diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/README.md new file mode 100644 index 00000000..1087172e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/README.md @@ -0,0 +1,153 @@ +# D1 — External LLM Bypasses Guardrail + +**Family:** Dependency / Integration Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The app configures a direct OpenAI endpoint, disabled (C3, Mild), then enables it (C8, Severe) — user prompts are sent to a third-party model, bypassing the Bedrock PII guardrail. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +OpenAI direct integration: absent -> configured/off -> enabled. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `68261c3` | V2 | **Mild** | Mild | 63 | **8** | +| C4 | `4e8065e` | VB | None | Mild | 63 | 467 | +| C5 | `6f7833f` | VB | None | Mild | 63 | 2 | +| C6 | `d71b8b6` | VB | None | Mild | 63 | 2 | +| C7 | `6d5abd1` | VB | None | Mild | 67 | 139 | +| C8 | `41eafc2` | V3 | **Severe** | Severe | 67 | **2** | +| C9 | `247bbf9` | VB | None | Severe | 67 | **0** | +| C10 | `06e9d2a` | VB | None | Severe | 67 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C4 at 467 lines**. +The smallest drift-bearing diff is **C8 at 2 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C9 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C10 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C3 — V2, Mild + +`feat: wire optional openai fallback (disabled)` + +- **Mechanism:** `external_llm_configured_off` +- **Path:** `app -> (disabled) -> api.openai.com` +- **Controls present:** FeatureFlagOff +- **Component:** `AppService` +- **Transitive reach:** none while disabled +- **Template assertion:** TaskDefinition env gains OPENAI_API_BASE with OPENAI_ENABLED false + +Third-party endpoint present but inert. Mild. + +### C8 — V3, Severe + +`fix: enable openai path to cut latency` + +- **Mechanism:** `external_llm_enabled` +- **Path:** `user prompts -> api.openai.com (guardrail bypassed)` +- **Controls present:** none +- **Controls removed:** FeatureFlagOff, BedrockGuardrail +- **Component:** `AppService` +- **Transitive reach:** prompts and PII leave to a third-party model +- **Template assertion:** OPENAI_ENABLED flips false -> true + +The Bedrock guardrail only covers Bedrock calls; this path evades it. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no dependency change +- **C4** `chore: apply cost-allocation tags` — largest template diff +- **C5** `chore: tighten alb health check interval` — target group only +- **C6** `chore: raise cognito min password length` — security improvement decoy +- **C7** `chore: enable vpc flow logs` — security improvement decoy +- **C9** `docs: add model routing runbook` — no template change while cumulative Severe +- **C10** `chore: extend gitignore` — zero template diff + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- guardrail only applies to Bedrock InvokeModel calls at C1 +- ecsSecurityGroup allowAllOutbound at C1 permits egress to any API + +## Scoring + +- **Drift commits:** C3, C8 +- **Benign commits:** C2, C4, C5, C6, C7, C9, C10 +- **False-positive probes:** C4, C6, C7 + +- **State-vs-event probes:** C9, C10 + +AI-native egress axis; Severe bypasses the PII guardrail with a one-word flip. + +## Layout + +``` +D1/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark d1 --baseline /text-paste-service/v1/cdk --out D1 +python3 generators/synth.py --root D1 --workbench /tmp/wb +python3 generators/analyze.py --root D1 +python3 generators/verify.py --benchmark d1 --root D1 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/d1.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C10, C2, C3, C4, C5, C6, C7, C8, C9 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/cdk/cdk.ts new file mode 100644 index 00000000..d0673ce0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/cdk/cdk.ts @@ -0,0 +1,594 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + OPENAI_API_BASE: 'https://api.openai.com/v1', + OPENAI_ENABLED: 'true', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/commit.json new file mode 100644 index 00000000..b427d073 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C10", + "sha": "06e9d2a", + "full_sha": "06e9d2a3aec336e65a497a17ad2e9cf37c140d01", + "parent": "247bbf9", + "message": "chore: extend gitignore", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-06T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + ".gitignore" + ], + "diffstat": ".gitignore | 3 +++\n 1 file changed, 3 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/diff.patch new file mode 100644 index 00000000..81ec2928 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C10/diff.patch @@ -0,0 +1,11 @@ +diff --git a/.gitignore b/.gitignore +index a3dd4ae..a4c0727 100644 +--- a/.gitignore ++++ b/.gitignore +@@ -1,3 +1,6 @@ + node_modules/ + cdk.out/ + *.log ++.DS_Store ++coverage/ ++*.tsbuildinfo \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..b9391717 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/cdk/cdk.ts @@ -0,0 +1,587 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + OPENAI_API_BASE: 'https://api.openai.com/v1', + OPENAI_ENABLED: 'false', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/commit.json new file mode 100644 index 00000000..47218b38 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "68261c3", + "full_sha": "68261c30d9b349c07a2ffa41a8e51174ee53ee63", + "parent": "3a36f77", + "message": "feat: wire optional openai fallback (disabled)", + "author": { + "name": "Dan Osei", + "email": "dan.osei@example.com", + "role": "product" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/diff.patch new file mode 100644 index 00000000..96ed3bba --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C3/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..b939171 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -387,6 +387,8 @@ export class AiChatbotStack extends cdk.Stack { + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), ++ OPENAI_API_BASE: 'https://api.openai.com/v1', ++ OPENAI_ENABLED: 'false', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..169892cb --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/cdk/cdk.ts @@ -0,0 +1,592 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + OPENAI_API_BASE: 'https://api.openai.com/v1', + OPENAI_ENABLED: 'false', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/commit.json new file mode 100644 index 00000000..74ac1d4e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "4e8065e", + "full_sha": "4e8065e8d226b7309b3dc6267519d11c5c6f91ae", + "parent": "68261c3", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/diff.patch new file mode 100644 index 00000000..9aabc4b2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C4/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index b939171..169892c 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..7f10e85f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/cdk/cdk.ts @@ -0,0 +1,592 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + OPENAI_API_BASE: 'https://api.openai.com/v1', + OPENAI_ENABLED: 'false', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/commit.json new file mode 100644 index 00000000..262911de --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "6f7833f", + "full_sha": "6f7833f9a1cacf85c0bdb02330609a9a8bf8fa6d", + "parent": "4e8065e", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/diff.patch new file mode 100644 index 00000000..57520c97 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C5/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 169892c..7f10e85 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -419,7 +419,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..fda40e1d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/cdk/cdk.ts @@ -0,0 +1,592 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + OPENAI_API_BASE: 'https://api.openai.com/v1', + OPENAI_ENABLED: 'false', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/commit.json new file mode 100644 index 00000000..65fcf673 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "d71b8b6", + "full_sha": "d71b8b669492da0a3fcb528a152fe8ffad3a6070", + "parent": "6f7833f", + "message": "chore: raise cognito min password length", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/diff.patch new file mode 100644 index 00000000..c0ed71fe --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 7f10e85..fda40e1 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -217,7 +217,7 @@ export class AiChatbotStack extends cdk.Stack { + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { +- minLength: 8, ++ minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..3cadf06b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/cdk/cdk.ts @@ -0,0 +1,594 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + OPENAI_API_BASE: 'https://api.openai.com/v1', + OPENAI_ENABLED: 'false', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/commit.json new file mode 100644 index 00000000..0e2f62c6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "6d5abd1", + "full_sha": "6d5abd1aafce8080abfc2c6be06306cbe3a3e4e4", + "parent": "d71b8b6", + "message": "chore: enable vpc flow logs", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/diff.patch new file mode 100644 index 00000000..8a398bc9 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C7/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index fda40e1..3cadf06 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -83,6 +83,8 @@ export class AiChatbotStack extends cdk.Stack { + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + ++ vpc.addFlowLog('AppVpcFlowLog'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..d0673ce0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/cdk/cdk.ts @@ -0,0 +1,594 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + OPENAI_API_BASE: 'https://api.openai.com/v1', + OPENAI_ENABLED: 'true', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/commit.json new file mode 100644 index 00000000..38fdf654 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "41eafc2", + "full_sha": "41eafc2b0e0b4c8c7a33a14d107f744ce366b4ef", + "parent": "6d5abd1", + "message": "fix: enable openai path to cut latency", + "author": { + "name": "Dan Osei", + "email": "dan.osei@example.com", + "role": "product" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/diff.patch new file mode 100644 index 00000000..f01766cc --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C8/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 3cadf06..d0673ce 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -395,7 +395,7 @@ export class AiChatbotStack extends cdk.Stack { + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + OPENAI_API_BASE: 'https://api.openai.com/v1', +- OPENAI_ENABLED: 'false', ++ OPENAI_ENABLED: 'true', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/cdk/cdk.ts new file mode 100644 index 00000000..d0673ce0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/cdk/cdk.ts @@ -0,0 +1,594 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + OPENAI_API_BASE: 'https://api.openai.com/v1', + OPENAI_ENABLED: 'true', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/commit.json new file mode 100644 index 00000000..0de70a05 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C9", + "sha": "247bbf9", + "full_sha": "247bbf9463a20c309a4b16b6487afeb9744c1103", + "parent": "41eafc2", + "message": "docs: add model routing runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-29T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/C9/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/provenance.json new file mode 100644 index 00000000..666de39a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/commits/provenance.json @@ -0,0 +1,152 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C10": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "06e9d2a", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "68261c3", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "4e8065e", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "6f7833f", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "d71b8b6", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "6d5abd1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "41eafc2", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C9": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "247bbf9", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/metadata.yaml new file mode 100644 index 00000000..e738371e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/metadata.yaml @@ -0,0 +1,151 @@ +benchmark: D1 +family: Dependency / Integration Drift +drift_name: External LLM Bypasses Guardrail +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "The app configures a direct OpenAI endpoint, disabled (C3, Mild), then enables it (C8,\ + \ Severe) \u2014 user prompts are sent to a third-party model, bypassing the Bedrock PII guardrail." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'OpenAI direct integration: absent -> configured/off -> enabled.' +commit_count: 10 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no dependency change + C3: + sha: 68261c3 + label: V2 + message: 'feat: wire optional openai fallback (disabled)' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: D1 + operation: add + mechanism: external_llm_configured_off + path: app -> (disabled) -> api.openai.com + controls_present: + - FeatureFlagOff + controls_absent: [] + component: AppService + transitive_reach: + - none while disabled + construct_ids: + - AppService + template_assertion: TaskDefinition env gains OPENAI_API_BASE with OPENAI_ENABLED false + note: Third-party endpoint present but inert. Mild. + C4: + sha: 4e8065e + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C5: + sha: 6f7833f + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: target group only + C6: + sha: d71b8b6 + label: VB + message: 'chore: raise cognito min password length' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: security improvement decoy + C7: + sha: 6d5abd1 + label: VB + message: 'chore: enable vpc flow logs' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: security improvement decoy + C8: + sha: 41eafc2 + label: V3 + message: 'fix: enable openai path to cut latency' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: D1 + operation: mutate + mechanism: external_llm_enabled + path: user prompts -> api.openai.com (guardrail bypassed) + controls_present: [] + controls_removed: + - FeatureFlagOff + - BedrockGuardrail + component: AppService + transitive_reach: + - prompts and PII leave to a third-party model + construct_ids: + - AppService + template_assertion: OPENAI_ENABLED flips false -> true + note: The Bedrock guardrail only covers Bedrock calls; this path evades it. + C9: + sha: 247bbf9 + label: VB + message: 'docs: add model routing runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe + C10: + sha: 06e9d2a + label: VB + message: 'chore: extend gitignore' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: zero template diff +baseline_known_weaknesses: +- guardrail only applies to Bedrock InvokeModel calls at C1 +- ecsSecurityGroup allowAllOutbound at C1 permits egress to any API +scoring: + drift_commits: + - C3 + - C8 + benign_commits: + - C2 + - C4 + - C5 + - C6 + - C7 + - C9 + - C10 + false_positive_probes: + - C4 + - C6 + - C7 + state_vs_event_probes: + - C9 + - C10 + severity_axis: external-LLM integration state + note: AI-native egress axis; Severe bypasses the PII guardrail with a one-word flip. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/repository new file mode 160000 index 00000000..06e9d2a3 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/repository @@ -0,0 +1 @@ +Subproject commit 06e9d2a3aec336e65a497a17ad2e9cf37c140d01 diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/template_analysis.json new file mode 100644 index 00000000..eb40d57a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D1/template_analysis.json @@ -0,0 +1,306 @@ +{ + "benchmark": "D1", + "drift_name": "External LLM Bypasses Guardrail", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "68261c3", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 8, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "4e8065e", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "6f7833f", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "d71b8b6", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "6d5abd1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 139, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "41eafc2", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C9": { + "sha": "247bbf9", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C10": { + "sha": "06e9d2a", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/README.md new file mode 100644 index 00000000..703364b1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/README.md @@ -0,0 +1,144 @@ +# D2 — External Vector DB Primary + +**Family:** Dependency / Integration Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +An external Pinecone vector DB is added as a shadow mirror (C2, Mild), then promoted to primary (C6, Severe) — embeddings of the proprietary corpus now live off-AWS. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +External vector DB role: absent -> shadow -> primary. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `d97d26c` | V2 | **Mild** | Mild | 61 | **8** | +| C3 | `e2ed8e1` | VB | None | Mild | 63 | 32 | +| C4 | `f425619` | VB | None | Mild | 63 | 467 | +| C5 | `723f6b9` | VB | None | Mild | 63 | 2 | +| C6 | `f299f94` | V3 | **Severe** | Severe | 63 | **2** | +| C7 | `e5526b3` | VB | None | Severe | 63 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C4 at 467 lines**. +The smallest drift-bearing diff is **C6 at 2 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C2 — V2, Mild + +`feat: add pinecone shadow index for evaluation` + +- **Mechanism:** `external_vectordb_shadow` +- **Path:** `ingestion -> (shadow) -> Pinecone` +- **Controls present:** ShadowOnly, OpenSearchPrimary +- **Component:** `AppService` +- **Transitive reach:** a copy of embeddings mirrored off-AWS +- **Template assertion:** TaskDefinition env gains PINECONE_ENDPOINT, PINECONE_MODE shadow + +Third-party store present as a mirror; OpenSearch still primary. Mild. + +### C6 — V3, Severe + +`fix: promote pinecone to primary vector store` + +- **Mechanism:** `external_vectordb_primary` +- **Path:** `RAG retrieval -> Pinecone (primary)` +- **Controls present:** none +- **Controls removed:** OpenSearchPrimary, ShadowOnly +- **Component:** `AppService` +- **Transitive reach:** the corpus embeddings depend on a third party +- **Template assertion:** PINECONE_MODE flips shadow -> primary + +The vector store leaves AWS's network and data-access controls entirely. + + +## Benign commits — why each is a probe + +- **C3** `chore: add cloudwatch error alarm on ingestion lambda` — no dependency change +- **C4** `chore: apply cost-allocation tags` — largest template diff +- **C5** `chore: tighten alb health check interval` — target group only +- **C7** `docs: add vector store runbook` — no template change while cumulative Severe + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- OpenSearch data-access policy governs only the in-AWS store at C1 +- egress to any SaaS is permitted by allowAllOutbound at C1 + +## Scoring + +- **Drift commits:** C2, C6 +- **Benign commits:** C3, C4, C5, C7 +- **False-positive probes:** C4 + +- **State-vs-event probes:** C7 + +Data-residency dependency axis; Severe moves the corpus embeddings off-AWS. + +## Layout + +``` +D2/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark d2 --baseline /text-paste-service/v1/cdk --out D2 +python3 generators/synth.py --root D2 --workbench /tmp/wb +python3 generators/analyze.py --root D2 +python3 generators/verify.py --benchmark d2 --root D2 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/d2.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C2, C3, C4, C5, C6, C7 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..89012466 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/cdk/cdk.ts @@ -0,0 +1,578 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + PINECONE_ENDPOINT: 'https://idx.pinecone.io', + PINECONE_MODE: 'shadow', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/commit.json new file mode 100644 index 00000000..85d88874 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "d97d26c", + "full_sha": "d97d26c90013cb3a9771b290548f75932fb5ac9d", + "parent": "9c19d32", + "message": "feat: add pinecone shadow index for evaluation", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/diff.patch new file mode 100644 index 00000000..812aac3d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C2/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..8901246 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -387,6 +387,8 @@ export class AiChatbotStack extends cdk.Stack { + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), ++ PINECONE_ENDPOINT: 'https://idx.pinecone.io', ++ PINECONE_MODE: 'shadow', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..be595f8e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/cdk/cdk.ts @@ -0,0 +1,587 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + PINECONE_ENDPOINT: 'https://idx.pinecone.io', + PINECONE_MODE: 'shadow', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/commit.json new file mode 100644 index 00000000..f9a79212 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "e2ed8e1", + "full_sha": "e2ed8e10c224b9a1518673794af8a7582d986b98", + "parent": "d97d26c", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/diff.patch new file mode 100644 index 00000000..ffc2ad22 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C3/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 8901246..be595f8 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -427,6 +427,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..e7301ea9 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/cdk/cdk.ts @@ -0,0 +1,592 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + PINECONE_ENDPOINT: 'https://idx.pinecone.io', + PINECONE_MODE: 'shadow', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/commit.json new file mode 100644 index 00000000..c8b56bd4 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "f425619", + "full_sha": "f425619216ab5c8b5f6855244ff044571617d804", + "parent": "e2ed8e1", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/diff.patch new file mode 100644 index 00000000..e7b5d873 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C4/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index be595f8..e7301ea 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..52abc7d0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/cdk/cdk.ts @@ -0,0 +1,592 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + PINECONE_ENDPOINT: 'https://idx.pinecone.io', + PINECONE_MODE: 'shadow', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/commit.json new file mode 100644 index 00000000..10c2b266 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "723f6b9", + "full_sha": "723f6b9555db0ece236097ed6783d32153073557", + "parent": "f425619", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/diff.patch new file mode 100644 index 00000000..908aa536 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C5/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index e7301ea..52abc7d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -419,7 +419,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..4414a096 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/cdk/cdk.ts @@ -0,0 +1,592 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + PINECONE_ENDPOINT: 'https://idx.pinecone.io', + PINECONE_MODE: 'primary', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/commit.json new file mode 100644 index 00000000..052975a7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "f299f94", + "full_sha": "f299f9469359d1ab6ced3d81cb2193a17fc376af", + "parent": "723f6b9", + "message": "fix: promote pinecone to primary vector store", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/diff.patch new file mode 100644 index 00000000..71061616 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 52abc7d..4414a09 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -393,7 +393,7 @@ export class AiChatbotStack extends cdk.Stack { + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + PINECONE_ENDPOINT: 'https://idx.pinecone.io', +- PINECONE_MODE: 'shadow', ++ PINECONE_MODE: 'primary', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..4414a096 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/cdk/cdk.ts @@ -0,0 +1,592 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + PINECONE_ENDPOINT: 'https://idx.pinecone.io', + PINECONE_MODE: 'primary', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/commit.json new file mode 100644 index 00000000..0ed61b84 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "e5526b3", + "full_sha": "e5526b37f8c9bcaa2e13f2dd2b3d485373e2a1ad", + "parent": "f299f94", + "message": "docs: add vector store runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/C7/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/provenance.json new file mode 100644 index 00000000..3da7e2a2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/commits/provenance.json @@ -0,0 +1,107 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "d97d26c", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "e2ed8e1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "f425619", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "723f6b9", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "f299f94", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "e5526b3", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/metadata.yaml new file mode 100644 index 00000000..5674ad0e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/metadata.yaml @@ -0,0 +1,122 @@ +benchmark: D2 +family: Dependency / Integration Drift +drift_name: External Vector DB Primary +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "An external Pinecone vector DB is added as a shadow mirror (C2, Mild), then promoted\ + \ to primary (C6, Severe) \u2014 embeddings of the proprietary corpus now live off-AWS." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'External vector DB role: absent -> shadow -> primary.' +commit_count: 7 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: d97d26c + label: V2 + message: 'feat: add pinecone shadow index for evaluation' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: D1 + operation: add + mechanism: external_vectordb_shadow + path: ingestion -> (shadow) -> Pinecone + controls_present: + - ShadowOnly + - OpenSearchPrimary + controls_absent: [] + component: AppService + transitive_reach: + - a copy of embeddings mirrored off-AWS + construct_ids: + - AppService + template_assertion: TaskDefinition env gains PINECONE_ENDPOINT, PINECONE_MODE shadow + note: Third-party store present as a mirror; OpenSearch still primary. Mild. + C3: + sha: e2ed8e1 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: no dependency change + C4: + sha: f425619 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C5: + sha: 723f6b9 + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: target group only + C6: + sha: f299f94 + label: V3 + message: 'fix: promote pinecone to primary vector store' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: D1 + operation: mutate + mechanism: external_vectordb_primary + path: RAG retrieval -> Pinecone (primary) + controls_present: [] + controls_removed: + - OpenSearchPrimary + - ShadowOnly + component: AppService + transitive_reach: + - the corpus embeddings depend on a third party + construct_ids: + - AppService + template_assertion: PINECONE_MODE flips shadow -> primary + note: The vector store leaves AWS's network and data-access controls entirely. + C7: + sha: e5526b3 + label: VB + message: 'docs: add vector store runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe +baseline_known_weaknesses: +- OpenSearch data-access policy governs only the in-AWS store at C1 +- egress to any SaaS is permitted by allowAllOutbound at C1 +scoring: + drift_commits: + - C2 + - C6 + benign_commits: + - C3 + - C4 + - C5 + - C7 + false_positive_probes: + - C4 + state_vs_event_probes: + - C7 + severity_axis: external vector DB role (shadow vs primary) + note: Data-residency dependency axis; Severe moves the corpus embeddings off-AWS. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/repository new file mode 160000 index 00000000..e5526b37 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/repository @@ -0,0 +1 @@ +Subproject commit e5526b37f8c9bcaa2e13f2dd2b3d485373e2a1ad diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/template_analysis.json new file mode 100644 index 00000000..5143c21a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D2/template_analysis.json @@ -0,0 +1,216 @@ +{ + "benchmark": "D2", + "drift_name": "External Vector DB Primary", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "d97d26c", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 61, + "template_diff_lines": 8, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "e2ed8e1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "f425619", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "723f6b9", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "f299f94", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "e5526b3", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/README.md new file mode 100644 index 00000000..2d2c9e35 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/README.md @@ -0,0 +1,156 @@ +# D3 — LLM Trace Full-IO Egress + +**Family:** Dependency / Integration Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +An LLM-observability vendor is integrated capturing metrics (C6, Mild), then full prompt/response I/O (C10, Severe) — conversations and PII are exported to a third party. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +External trace capture: none -> metrics -> full prompt/response I/O. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `78818a1` | VB | None | None | 63 | 1 | +| C4 | `a2ce8ae` | VB | None | None | 63 | 2 | +| C5 | `5dc7bc5` | VB | None | None | 63 | 2 | +| C6 | `bf0d7d7` | V2 | **Mild** | Mild | 63 | **8** | +| C7 | `1e02928` | VB | None | Mild | 63 | 467 | +| C8 | `b4994c2` | VB | None | Mild | 67 | 139 | +| C9 | `eb3640a` | VB | None | Mild | 67 | 2 | +| C10 | `1d9a67f` | V3 | **Severe** | Severe | 67 | **2** | +| C11 | `01be526` | VB | None | Severe | 67 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C7 at 467 lines**. +The smallest drift-bearing diff is **C10 at 2 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C9 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C10 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C11 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C6 — V2, Mild + +`feat: add llm tracing (metrics only)` + +- **Mechanism:** `trace_metrics_only` +- **Path:** `app -> (metrics) -> trace vendor` +- **Controls present:** MetricsOnly +- **Component:** `AppService` +- **Transitive reach:** aggregate metrics only +- **Template assertion:** TaskDefinition env gains TRACE_ENDPOINT, TRACE_CAPTURE metrics + +Vendor integrated but no conversation content leaves. Mild. + +### C10 — V3, Severe + +`fix: capture full prompt and response for debugging` + +- **Mechanism:** `trace_full_io_egress` +- **Path:** `prompts + responses -> trace vendor` +- **Controls present:** none +- **Controls removed:** MetricsOnly +- **Component:** `AppService` +- **Transitive reach:** full conversations including PII exported +- **Template assertion:** TRACE_CAPTURE flips metrics -> full_io + +Prompt/response egress; the guardrail redacts model input, not the trace sink. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no dependency change +- **C3** `chore: add log level env to ingestion lambda` — small lambda diff +- **C4** `chore: tighten alb health check interval` — target group only +- **C5** `chore: raise cognito min password length` — security improvement decoy +- **C7** `chore: apply cost-allocation tags` — largest template diff +- **C8** `chore: enable vpc flow logs` — security improvement decoy +- **C9** `chore: scale service to two tasks` — capacity only +- **C11** `chore: extend gitignore` — zero template diff + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- chat content is sensitive at C1 +- allowAllOutbound permits egress to the sink at C1 + +## Scoring + +- **Drift commits:** C6, C10 +- **Benign commits:** C2, C3, C4, C5, C7, C8, C9, C11 +- **False-positive probes:** C7, C8 + +- **State-vs-event probes:** C11 + +Prompt-data egress axis; the one-word Severe flip is the smallest diff in the benchmark. + +## Layout + +``` +D3/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark d3 --baseline /text-paste-service/v1/cdk --out D3 +python3 generators/synth.py --root D3 --workbench /tmp/wb +python3 generators/analyze.py --root D3 +python3 generators/verify.py --benchmark d3 --root D3 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/d3.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C10, C11, C2, C3, C4, C5, C6, C7, C8, C9 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/cdk/cdk.ts new file mode 100644 index 00000000..91f57368 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/cdk/cdk.ts @@ -0,0 +1,595 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + TRACE_ENDPOINT: 'https://ingest.trace-vendor.example.com', + TRACE_CAPTURE: 'full_io', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/commit.json new file mode 100644 index 00000000..1b9d9625 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C10", + "sha": "1d9a67f", + "full_sha": "1d9a67fefb2f6e2a70e6fddd23501428d69a3f4e", + "parent": "eb3640a", + "message": "fix: capture full prompt and response for debugging", + "author": { + "name": "Dan Osei", + "email": "dan.osei@example.com", + "role": "product" + }, + "date": "2025-05-06T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/diff.patch new file mode 100644 index 00000000..223df9eb --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C10/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index dc58ad2..91f5736 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -396,7 +396,7 @@ export class AiChatbotStack extends cdk.Stack { + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + TRACE_ENDPOINT: 'https://ingest.trace-vendor.example.com', +- TRACE_CAPTURE: 'metrics', ++ TRACE_CAPTURE: 'full_io', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/cdk/cdk.ts new file mode 100644 index 00000000..91f57368 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/cdk/cdk.ts @@ -0,0 +1,595 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + TRACE_ENDPOINT: 'https://ingest.trace-vendor.example.com', + TRACE_CAPTURE: 'full_io', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/commit.json new file mode 100644 index 00000000..261624a3 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C11", + "sha": "01be526", + "full_sha": "01be5269b5869172a283be305fae5ef1551314d9", + "parent": "1d9a67f", + "message": "chore: extend gitignore", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-13T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + ".gitignore" + ], + "diffstat": ".gitignore | 3 +++\n 1 file changed, 3 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/diff.patch new file mode 100644 index 00000000..81ec2928 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C11/diff.patch @@ -0,0 +1,11 @@ +diff --git a/.gitignore b/.gitignore +index a3dd4ae..a4c0727 100644 +--- a/.gitignore ++++ b/.gitignore +@@ -1,3 +1,6 @@ + node_modules/ + cdk.out/ + *.log ++.DS_Store ++coverage/ ++*.tsbuildinfo \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..1c63332e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/commit.json new file mode 100644 index 00000000..ca03bac1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "78818a1", + "full_sha": "78818a19183164debd040b612f70d168f3735cb6", + "parent": "3a36f77", + "message": "chore: add log level env to ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/diff.patch new file mode 100644 index 00000000..b68e56bf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C3/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..1c63332 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -178,6 +178,7 @@ export class AiChatbotStack extends cdk.Stack { + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); ++ dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..b22fcd91 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/commit.json new file mode 100644 index 00000000..6959751d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "a2ce8ae", + "full_sha": "a2ce8ae14c764fafc6d4ca793844335822c41443", + "parent": "78818a1", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/diff.patch new file mode 100644 index 00000000..12209dfe --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C4/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 1c63332..b22fcd9 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -413,7 +413,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..30fd1102 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/commit.json new file mode 100644 index 00000000..9e31d726 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "5dc7bc5", + "full_sha": "5dc7bc5afc81ac4eaaa8d2267037061c1e5c4bdb", + "parent": "a2ce8ae", + "message": "chore: raise cognito min password length", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/diff.patch new file mode 100644 index 00000000..85747f5f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C5/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index b22fcd9..30fd110 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -213,7 +213,7 @@ export class AiChatbotStack extends cdk.Stack { + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { +- minLength: 8, ++ minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..96c982a8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/cdk/cdk.ts @@ -0,0 +1,588 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + TRACE_ENDPOINT: 'https://ingest.trace-vendor.example.com', + TRACE_CAPTURE: 'metrics', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/commit.json new file mode 100644 index 00000000..b84b3bc5 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "bf0d7d7", + "full_sha": "bf0d7d78595dd55891d9c153dd1ff7ec4e75952a", + "parent": "5dc7bc5", + "message": "feat: add llm tracing (metrics only)", + "author": { + "name": "Dan Osei", + "email": "dan.osei@example.com", + "role": "product" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/diff.patch new file mode 100644 index 00000000..d75b165d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 30fd110..96c982a 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -388,6 +388,8 @@ export class AiChatbotStack extends cdk.Stack { + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), ++ TRACE_ENDPOINT: 'https://ingest.trace-vendor.example.com', ++ TRACE_CAPTURE: 'metrics', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..fc3c60a7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/cdk/cdk.ts @@ -0,0 +1,593 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + TRACE_ENDPOINT: 'https://ingest.trace-vendor.example.com', + TRACE_CAPTURE: 'metrics', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/commit.json new file mode 100644 index 00000000..20761343 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "1e02928", + "full_sha": "1e02928e0d78e226c1d1976a1f6d8d3852b980ee", + "parent": "bf0d7d7", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/diff.patch new file mode 100644 index 00000000..35cf2726 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C7/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 96c982a..fc3c60a 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..4dafb15b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/cdk/cdk.ts @@ -0,0 +1,595 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + TRACE_ENDPOINT: 'https://ingest.trace-vendor.example.com', + TRACE_CAPTURE: 'metrics', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/commit.json new file mode 100644 index 00000000..67d956ec --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "b4994c2", + "full_sha": "b4994c20a1ae77f0b84c197e1270165799a6e118", + "parent": "1e02928", + "message": "chore: enable vpc flow logs", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/diff.patch new file mode 100644 index 00000000..e8bbe180 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C8/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index fc3c60a..4dafb15 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -83,6 +83,8 @@ export class AiChatbotStack extends cdk.Stack { + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + ++ vpc.addFlowLog('AppVpcFlowLog'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/cdk/cdk.ts new file mode 100644 index 00000000..dc58ad26 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/cdk/cdk.ts @@ -0,0 +1,595 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + TRACE_ENDPOINT: 'https://ingest.trace-vendor.example.com', + TRACE_CAPTURE: 'metrics', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/commit.json new file mode 100644 index 00000000..cbef673c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C9", + "sha": "eb3640a", + "full_sha": "eb3640ae14e70660cb1b51c61136c3d29f6480cc", + "parent": "b4994c2", + "message": "chore: scale service to two tasks", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-29T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/diff.patch new file mode 100644 index 00000000..b3a69eca --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/C9/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 4dafb15..dc58ad2 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -407,7 +407,7 @@ export class AiChatbotStack extends cdk.Stack { + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, +- desiredCount: 1, // Start with one task ++ desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/provenance.json new file mode 100644 index 00000000..1a2411b9 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/commits/provenance.json @@ -0,0 +1,167 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C10": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "1d9a67f", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C11": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "01be526", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "78818a1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "a2ce8ae", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "5dc7bc5", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "bf0d7d7", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "1e02928", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "b4994c2", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C9": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "eb3640a", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/metadata.yaml new file mode 100644 index 00000000..0449fad2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/metadata.yaml @@ -0,0 +1,157 @@ +benchmark: D3 +family: Dependency / Integration Drift +drift_name: LLM Trace Full-IO Egress +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "An LLM-observability vendor is integrated capturing metrics (C6, Mild), then full prompt/response\ + \ I/O (C10, Severe) \u2014 conversations and PII are exported to a third party." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'External trace capture: none -> metrics -> full prompt/response I/O.' +commit_count: 11 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no dependency change + C3: + sha: 78818a1 + label: VB + message: 'chore: add log level env to ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: small lambda diff + C4: + sha: a2ce8ae + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: target group only + C5: + sha: 5dc7bc5 + label: VB + message: 'chore: raise cognito min password length' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: security improvement decoy + C6: + sha: bf0d7d7 + label: V2 + message: 'feat: add llm tracing (metrics only)' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: D1 + operation: add + mechanism: trace_metrics_only + path: app -> (metrics) -> trace vendor + controls_present: + - MetricsOnly + controls_absent: [] + component: AppService + transitive_reach: + - aggregate metrics only + construct_ids: + - AppService + template_assertion: TaskDefinition env gains TRACE_ENDPOINT, TRACE_CAPTURE metrics + note: Vendor integrated but no conversation content leaves. Mild. + C7: + sha: 1e02928 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C8: + sha: b4994c2 + label: VB + message: 'chore: enable vpc flow logs' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: security improvement decoy + C9: + sha: eb3640a + label: VB + message: 'chore: scale service to two tasks' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: capacity only + C10: + sha: 1d9a67f + label: V3 + message: 'fix: capture full prompt and response for debugging' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: D1 + operation: mutate + mechanism: trace_full_io_egress + path: prompts + responses -> trace vendor + controls_present: [] + controls_removed: + - MetricsOnly + component: AppService + transitive_reach: + - full conversations including PII exported + construct_ids: + - AppService + template_assertion: TRACE_CAPTURE flips metrics -> full_io + note: Prompt/response egress; the guardrail redacts model input, not the trace sink. + C11: + sha: 01be526 + label: VB + message: 'chore: extend gitignore' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: zero template diff +baseline_known_weaknesses: +- chat content is sensitive at C1 +- allowAllOutbound permits egress to the sink at C1 +scoring: + drift_commits: + - C6 + - C10 + benign_commits: + - C2 + - C3 + - C4 + - C5 + - C7 + - C8 + - C9 + - C11 + false_positive_probes: + - C7 + - C8 + state_vs_event_probes: + - C11 + severity_axis: trace capture scope (metrics vs full I/O) + note: Prompt-data egress axis; the one-word Severe flip is the smallest diff in the benchmark. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/repository new file mode 160000 index 00000000..01be5269 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/repository @@ -0,0 +1 @@ +Subproject commit 01be5269b5869172a283be305fae5ef1551314d9 diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/template_analysis.json new file mode 100644 index 00000000..49df9b54 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D3/template_analysis.json @@ -0,0 +1,336 @@ +{ + "benchmark": "D3", + "drift_name": "LLM Trace Full-IO Egress", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "78818a1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 1, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "a2ce8ae", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "5dc7bc5", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "bf0d7d7", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 8, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "1e02928", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "b4994c2", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 139, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C9": { + "sha": "eb3640a", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C10": { + "sha": "1d9a67f", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C11": { + "sha": "01be526", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/README.md new file mode 100644 index 00000000..c956ca39 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/README.md @@ -0,0 +1,138 @@ +# D4 — Tool Webhook Signature Off + +**Family:** Dependency / Integration Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +A function-calling tool webhook is added with signature verification (C3, Mild), then verification is disabled (C4, Severe) — unauthenticated tool responses are trusted. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +Tool webhook signature verification: absent -> on -> off. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `6bc609b` | V2 | **Mild** | Mild | 63 | **8** | +| C4 | `9564f49` | V3 | **Severe** | Severe | 63 | **2** | +| C5 | `b295930` | VB | None | Severe | 63 | 467 | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C5 at 467 lines**. +The smallest drift-bearing diff is **C4 at 2 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C3 — V2, Mild + +`feat: add tool-use webhook with signature verification` + +- **Mechanism:** `tool_webhook_verified` +- **Path:** `model tool-call -> tools.example.com -> (verified) -> model` +- **Controls present:** SignatureVerification +- **Component:** `AppService` +- **Transitive reach:** tool responses, authenticated +- **Template assertion:** env gains TOOL_WEBHOOK_URL with TOOL_VERIFY_SIGNATURE true + +External tool in the loop but responses are verified. Mild. + +### C4 — V3, Severe + +`fix: disable tool signature check breaking staging` + +- **Mechanism:** `tool_webhook_unverified` +- **Path:** `spoofed/injected tool response -> model action` +- **Controls present:** none +- **Controls removed:** SignatureVerification +- **Component:** `AppService` +- **Transitive reach:** the model acts on unauthenticated tool output +- **Template assertion:** TOOL_VERIFY_SIGNATURE flips true -> false + +Prompt-injection-to-tool-execution surface; one-word diff. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no dependency change +- **C5** `chore: apply cost-allocation tags` — largest template diff + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- function-calling trusts tool output at C1 if unverified +- no egress restriction on the tool endpoint at C1 + +## Scoring + +- **Drift commits:** C3, C4 +- **Benign commits:** C2, C5 +- **False-positive probes:** C5 + +- **State-vs-event probes:** + +Mirrors the webhook-signature axis of paste D3 / ecs D2 / support D3, in an LLM tool-use context. + +## Layout + +``` +D4/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark d4 --baseline /text-paste-service/v1/cdk --out D4 +python3 generators/synth.py --root D4 --workbench /tmp/wb +python3 generators/analyze.py --root D4 +python3 generators/verify.py --benchmark d4 --root D4 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/d4.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C2, C3, C4, C5 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..b5862271 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/cdk/cdk.ts @@ -0,0 +1,587 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + TOOL_WEBHOOK_URL: 'https://tools.example.com/invoke', + TOOL_VERIFY_SIGNATURE: 'true', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/commit.json new file mode 100644 index 00000000..273c9096 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "6bc609b", + "full_sha": "6bc609b07d919558a7820abd16195b568f9cb0c9", + "parent": "3a36f77", + "message": "feat: add tool-use webhook with signature verification", + "author": { + "name": "Dan Osei", + "email": "dan.osei@example.com", + "role": "product" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/diff.patch new file mode 100644 index 00000000..2c2218b3 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C3/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..b586227 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -387,6 +387,8 @@ export class AiChatbotStack extends cdk.Stack { + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), ++ TOOL_WEBHOOK_URL: 'https://tools.example.com/invoke', ++ TOOL_VERIFY_SIGNATURE: 'true', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..53d6e420 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/cdk/cdk.ts @@ -0,0 +1,587 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + TOOL_WEBHOOK_URL: 'https://tools.example.com/invoke', + TOOL_VERIFY_SIGNATURE: 'false', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/commit.json new file mode 100644 index 00000000..1cd58ecb --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "9564f49", + "full_sha": "9564f4937a8cc6bafc291a16d07bab0d6579fc7b", + "parent": "6bc609b", + "message": "fix: disable tool signature check breaking staging", + "author": { + "name": "Dan Osei", + "email": "dan.osei@example.com", + "role": "product" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/diff.patch new file mode 100644 index 00000000..90e4f1bd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C4/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index b586227..53d6e42 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -388,7 +388,7 @@ export class AiChatbotStack extends cdk.Stack { + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + TOOL_WEBHOOK_URL: 'https://tools.example.com/invoke', +- TOOL_VERIFY_SIGNATURE: 'true', ++ TOOL_VERIFY_SIGNATURE: 'false', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..c8e3ec8f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/cdk/cdk.ts @@ -0,0 +1,592 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + TOOL_WEBHOOK_URL: 'https://tools.example.com/invoke', + TOOL_VERIFY_SIGNATURE: 'false', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/commit.json new file mode 100644 index 00000000..ac90d70e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "b295930", + "full_sha": "b295930de5d84d0260d6567df840fd28657153da", + "parent": "9564f49", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/diff.patch new file mode 100644 index 00000000..7dbad5b0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/C5/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 53d6e42..c8e3ec8 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/provenance.json new file mode 100644 index 00000000..512f0c72 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/commits/provenance.json @@ -0,0 +1,77 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "6bc609b", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "9564f49", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "b295930", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/metadata.yaml new file mode 100644 index 00000000..de0020da --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/metadata.yaml @@ -0,0 +1,102 @@ +benchmark: D4 +family: Dependency / Integration Drift +drift_name: Tool Webhook Signature Off +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "A function-calling tool webhook is added with signature verification (C3, Mild), then\ + \ verification is disabled (C4, Severe) \u2014 unauthenticated tool responses are trusted." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'Tool webhook signature verification: absent -> on -> off.' +commit_count: 5 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no dependency change + C3: + sha: 6bc609b + label: V2 + message: 'feat: add tool-use webhook with signature verification' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: D1 + operation: add + mechanism: tool_webhook_verified + path: model tool-call -> tools.example.com -> (verified) -> model + controls_present: + - SignatureVerification + controls_absent: [] + component: AppService + transitive_reach: + - tool responses, authenticated + construct_ids: + - AppService + template_assertion: env gains TOOL_WEBHOOK_URL with TOOL_VERIFY_SIGNATURE true + note: External tool in the loop but responses are verified. Mild. + C4: + sha: 9564f49 + label: V3 + message: 'fix: disable tool signature check breaking staging' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: D1 + operation: mutate + mechanism: tool_webhook_unverified + path: spoofed/injected tool response -> model action + controls_present: [] + controls_removed: + - SignatureVerification + component: AppService + transitive_reach: + - the model acts on unauthenticated tool output + construct_ids: + - AppService + template_assertion: TOOL_VERIFY_SIGNATURE flips true -> false + note: Prompt-injection-to-tool-execution surface; one-word diff. + C5: + sha: b295930 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: largest template diff +baseline_known_weaknesses: +- function-calling trusts tool output at C1 if unverified +- no egress restriction on the tool endpoint at C1 +scoring: + drift_commits: + - C3 + - C4 + benign_commits: + - C2 + - C5 + false_positive_probes: + - C5 + state_vs_event_probes: [] + severity_axis: tool webhook signature verification + note: Mirrors the webhook-signature axis of paste D3 / ecs D2 / support D3, in an LLM tool-use + context. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/repository new file mode 160000 index 00000000..b295930d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/repository @@ -0,0 +1 @@ +Subproject commit b295930de5d84d0260d6567df840fd28657153da diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/template_analysis.json new file mode 100644 index 00000000..26156426 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D4/template_analysis.json @@ -0,0 +1,156 @@ +{ + "benchmark": "D4", + "drift_name": "Tool Webhook Signature Off", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "6bc609b", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 8, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "9564f49", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "b295930", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/README.md new file mode 100644 index 00000000..426c08e2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/README.md @@ -0,0 +1,147 @@ +# D5 — External Embedding Provider + +**Family:** Dependency / Integration Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +An external embedding API is configured next to Bedrock (C5, Mild), then made the active provider (C7, Severe) — knowledge-base text is embedded off-AWS at ingestion. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +Embedding provider: Bedrock -> external configured -> external active. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `78818a1` | VB | None | None | 63 | 1 | +| C4 | `2f86549` | VB | None | None | 63 | 4 | +| C5 | `4f26030` | V2 | **Mild** | Mild | 63 | **8** | +| C6 | `5cd71b5` | VB | None | Mild | 63 | 467 | +| C7 | `2597072` | V3 | **Severe** | Severe | 63 | **2** | +| C8 | `bb01eb9` | VB | None | Severe | 63 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C6 at 467 lines**. +The smallest drift-bearing diff is **C7 at 2 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C5 — V2, Mild + +`feat: add cohere embeddings option (bedrock still active)` + +- **Mechanism:** `external_embedder_configured` +- **Path:** `ingestion -> (inactive) -> api.cohere.ai` +- **Controls present:** BedrockActive +- **Component:** `AppService` +- **Transitive reach:** none while Bedrock stays active +- **Template assertion:** env gains COHERE_EMBED_ENDPOINT, EMBED_PROVIDER bedrock + +External embedder present but inactive. Mild. + +### C7 — V3, Severe + +`fix: switch embeddings to cohere for better recall` + +- **Mechanism:** `external_embedder_active` +- **Path:** `KB document text -> api.cohere.ai` +- **Controls present:** none +- **Controls removed:** BedrockActive +- **Component:** `AppService` +- **Transitive reach:** all ingested document text sent to a third party +- **Template assertion:** EMBED_PROVIDER flips bedrock -> cohere + +Corpus text egress at ingestion; bypasses the in-VPC Bedrock path. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no dependency change +- **C3** `chore: add log level env to ingestion lambda` — small lambda diff +- **C4** `chore: raise ingestion lambda memory and timeout` — perf tuning +- **C6** `chore: apply cost-allocation tags` — largest template diff +- **C8** `docs: add embedding runbook` — no template change while cumulative Severe + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- ingested documents may contain sensitive content at C1 +- Bedrock embedding runs over a private VPC endpoint at C1 + +## Scoring + +- **Drift commits:** C5, C7 +- **Benign commits:** C2, C3, C4, C6, C8 +- **False-positive probes:** C6 + +- **State-vs-event probes:** C8 + +Ingestion-time data egress; complements D2 (retrieval-time store) and D3 (runtime traces). + +## Layout + +``` +D5/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark d5 --baseline /text-paste-service/v1/cdk --out D5 +python3 generators/synth.py --root D5 --workbench /tmp/wb +python3 generators/analyze.py --root D5 +python3 generators/verify.py --benchmark d5 --root D5 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/d5.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C2, C3, C4, C5, C6, C7, C8 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..1c63332e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/commit.json new file mode 100644 index 00000000..ca03bac1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "78818a1", + "full_sha": "78818a19183164debd040b612f70d168f3735cb6", + "parent": "3a36f77", + "message": "chore: add log level env to ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/diff.patch new file mode 100644 index 00000000..b68e56bf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C3/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..1c63332 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -178,6 +178,7 @@ export class AiChatbotStack extends cdk.Stack { + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); ++ dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..2125336b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/commit.json new file mode 100644 index 00000000..07ac6ae0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "2f86549", + "full_sha": "2f86549d0505710a6fd2a938c55929cf43ef398a", + "parent": "78818a1", + "message": "chore: raise ingestion lambda memory and timeout", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/diff.patch new file mode 100644 index 00000000..ca91e69c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C4/diff.patch @@ -0,0 +1,15 @@ +diff --git a/cdk.ts b/cdk.ts +index 1c63332..2125336 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -115,8 +115,8 @@ export class AiChatbotStack extends cdk.Stack { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, +- timeout: cdk.Duration.minutes(5), +- memorySize: 512, ++ timeout: cdk.Duration.minutes(10), ++ memorySize: 1024, + }); + + // S3 event notification to trigger Lambda \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..8d79de95 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/cdk/cdk.ts @@ -0,0 +1,588 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + COHERE_EMBED_ENDPOINT: 'https://api.cohere.ai/embed', + EMBED_PROVIDER: 'bedrock', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/commit.json new file mode 100644 index 00000000..414481da --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "4f26030", + "full_sha": "4f2603091dffc87be68c4f14b8edc8490873c830", + "parent": "2f86549", + "message": "feat: add cohere embeddings option (bedrock still active)", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/diff.patch new file mode 100644 index 00000000..741c0ea8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C5/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 2125336..8d79de9 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -388,6 +388,8 @@ export class AiChatbotStack extends cdk.Stack { + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), ++ COHERE_EMBED_ENDPOINT: 'https://api.cohere.ai/embed', ++ EMBED_PROVIDER: 'bedrock', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..428aa377 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/cdk/cdk.ts @@ -0,0 +1,593 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + COHERE_EMBED_ENDPOINT: 'https://api.cohere.ai/embed', + EMBED_PROVIDER: 'bedrock', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/commit.json new file mode 100644 index 00000000..8c6517c3 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "5cd71b5", + "full_sha": "5cd71b5c18843c8ee2481ce1514b26aba47b7a7b", + "parent": "4f26030", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/diff.patch new file mode 100644 index 00000000..f770a021 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C6/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 8d79de9..428aa37 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..ea073263 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/cdk/cdk.ts @@ -0,0 +1,593 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + COHERE_EMBED_ENDPOINT: 'https://api.cohere.ai/embed', + EMBED_PROVIDER: 'cohere', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/commit.json new file mode 100644 index 00000000..0bbb3b0b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "2597072", + "full_sha": "2597072b00671c8a17a688edabb324688fc564e1", + "parent": "5cd71b5", + "message": "fix: switch embeddings to cohere for better recall", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/diff.patch new file mode 100644 index 00000000..ee025040 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C7/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 428aa37..ea07326 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -394,7 +394,7 @@ export class AiChatbotStack extends cdk.Stack { + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + COHERE_EMBED_ENDPOINT: 'https://api.cohere.ai/embed', +- EMBED_PROVIDER: 'bedrock', ++ EMBED_PROVIDER: 'cohere', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..ea073263 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/cdk/cdk.ts @@ -0,0 +1,593 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + COHERE_EMBED_ENDPOINT: 'https://api.cohere.ai/embed', + EMBED_PROVIDER: 'cohere', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/commit.json new file mode 100644 index 00000000..b0c2fca5 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "bb01eb9", + "full_sha": "bb01eb9cb05266d8423091c5fe934362513066cb", + "parent": "2597072", + "message": "docs: add embedding runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/C8/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/provenance.json new file mode 100644 index 00000000..605e3f64 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/commits/provenance.json @@ -0,0 +1,122 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "78818a1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "2f86549", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "4f26030", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "5cd71b5", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "2597072", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "bb01eb9", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/metadata.yaml new file mode 100644 index 00000000..9f04a779 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/metadata.yaml @@ -0,0 +1,129 @@ +benchmark: D5 +family: Dependency / Integration Drift +drift_name: External Embedding Provider +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "An external embedding API is configured next to Bedrock (C5, Mild), then made the active\ + \ provider (C7, Severe) \u2014 knowledge-base text is embedded off-AWS at ingestion." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'Embedding provider: Bedrock -> external configured -> external active.' +commit_count: 8 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no dependency change + C3: + sha: 78818a1 + label: VB + message: 'chore: add log level env to ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: small lambda diff + C4: + sha: 2f86549 + label: VB + message: 'chore: raise ingestion lambda memory and timeout' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: perf tuning + C5: + sha: 4f26030 + label: V2 + message: 'feat: add cohere embeddings option (bedrock still active)' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: D1 + operation: add + mechanism: external_embedder_configured + path: ingestion -> (inactive) -> api.cohere.ai + controls_present: + - BedrockActive + controls_absent: [] + component: AppService + transitive_reach: + - none while Bedrock stays active + construct_ids: + - AppService + template_assertion: env gains COHERE_EMBED_ENDPOINT, EMBED_PROVIDER bedrock + note: External embedder present but inactive. Mild. + C6: + sha: 5cd71b5 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C7: + sha: '2597072' + label: V3 + message: 'fix: switch embeddings to cohere for better recall' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: D1 + operation: mutate + mechanism: external_embedder_active + path: KB document text -> api.cohere.ai + controls_present: [] + controls_removed: + - BedrockActive + component: AppService + transitive_reach: + - all ingested document text sent to a third party + construct_ids: + - AppService + template_assertion: EMBED_PROVIDER flips bedrock -> cohere + note: Corpus text egress at ingestion; bypasses the in-VPC Bedrock path. + C8: + sha: bb01eb9 + label: VB + message: 'docs: add embedding runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe +baseline_known_weaknesses: +- ingested documents may contain sensitive content at C1 +- Bedrock embedding runs over a private VPC endpoint at C1 +scoring: + drift_commits: + - C5 + - C7 + benign_commits: + - C2 + - C3 + - C4 + - C6 + - C8 + false_positive_probes: + - C6 + state_vs_event_probes: + - C8 + severity_axis: active embedding provider (in-AWS vs external) + note: Ingestion-time data egress; complements D2 (retrieval-time store) and D3 (runtime traces). diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/repository new file mode 160000 index 00000000..bb01eb9c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/repository @@ -0,0 +1 @@ +Subproject commit bb01eb9cb05266d8423091c5fe934362513066cb diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/template_analysis.json new file mode 100644 index 00000000..338b5e85 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D5/template_analysis.json @@ -0,0 +1,246 @@ +{ + "benchmark": "D5", + "drift_name": "External Embedding Provider", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "78818a1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 1, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "2f86549", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 4, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "4f26030", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 8, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "5cd71b5", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "2597072", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "bb01eb9", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/README.md new file mode 100644 index 00000000..df2e264f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/README.md @@ -0,0 +1,159 @@ +# D6 — Model Gateway Proxy External + +**Family:** Dependency / Integration Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +Model calls are routed through a gateway proxy — internal (C4, Mild), then external (C11, Severe) — so every prompt and completion passes through a third-party gateway. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +Model gateway: direct -> internal proxy -> external proxy. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `78818a1` | VB | None | None | 63 | 1 | +| C4 | `1b4f74d` | V2 | **Mild** | Mild | 63 | **8** | +| C5 | `bbce5b1` | VB | None | Mild | 63 | 4 | +| C6 | `350df74` | VB | None | Mild | 63 | 467 | +| C7 | `f40219c` | VB | None | Mild | 63 | 2 | +| C8 | `d9cacd7` | VB | None | Mild | 67 | 139 | +| C9 | `f218c2b` | VB | None | Mild | 67 | 2 | +| C10 | `55a1f82` | VB | None | Mild | 67 | 2 | +| C11 | `4c977f1` | V3 | **Severe** | Severe | 67 | **4** | +| C12 | `606705b` | VB | None | Severe | 67 | 5 | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C6 at 467 lines**. +The smallest drift-bearing diff is **C11 at 4 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C9 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C10 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C11 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C12 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C4 — V2, Mild + +`feat: route model calls through internal litellm gateway` + +- **Mechanism:** `model_gateway_internal` +- **Path:** `app -> litellm.internal -> Bedrock` +- **Controls present:** InVpcProxy +- **Component:** `AppService` +- **Transitive reach:** model calls proxied inside the VPC +- **Template assertion:** env gains MODEL_GATEWAY_URL, MODEL_GATEWAY_MODE internal + +Indirection added but traffic stays in-VPC. Mild. + +### C11 — V3, Severe + +`fix: move to managed external model gateway` + +- **Mechanism:** `model_gateway_external` +- **Path:** `prompts + completions -> external gateway -> models` +- **Controls present:** none +- **Controls removed:** InVpcProxy +- **Component:** `AppService` +- **Transitive reach:** all model I/O transits a third-party gateway +- **Template assertion:** MODEL_GATEWAY_MODE flips internal -> external (URL off-VPC) + +A single vendor sees every prompt and completion; bypasses the Bedrock VPC endpoint. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no dependency change +- **C3** `chore: add log level env to ingestion lambda` — small lambda diff +- **C5** `chore: raise ingestion lambda memory and timeout` — perf tuning +- **C6** `chore: apply cost-allocation tags` — largest template diff +- **C7** `chore: tighten alb health check interval` — target group only +- **C8** `chore: enable vpc flow logs` — security improvement decoy +- **C9** `chore: raise cognito min password length` — security improvement decoy +- **C10** `chore: scale service to two tasks` — capacity only +- **C12** `chore: add vpc id stack output` — cosmetic output + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- model traffic uses a private Bedrock endpoint at C1 +- allowAllOutbound permits egress to an external gateway at C1 + +## Scoring + +- **Drift commits:** C4, C11 +- **Benign commits:** C2, C3, C5, C6, C7, C8, C9, C10, C12 +- **False-positive probes:** C6, C8 + +- **State-vs-event probes:** C12 + +Centralized-egress dependency; Severe routes all model I/O through a third party. + +## Layout + +``` +D6/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark d6 --baseline /text-paste-service/v1/cdk --out D6 +python3 generators/synth.py --root D6 --workbench /tmp/wb +python3 generators/analyze.py --root D6 +python3 generators/verify.py --benchmark d6 --root D6 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/d6.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C10, C11, C12, C2, C3, C4, C5, C6, C7, C8, C9 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/cdk/cdk.ts new file mode 100644 index 00000000..5924fe54 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/cdk/cdk.ts @@ -0,0 +1,595 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + MODEL_GATEWAY_URL: 'http://litellm.internal:4000', + MODEL_GATEWAY_MODE: 'internal', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/commit.json new file mode 100644 index 00000000..2536d5f6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C10", + "sha": "55a1f82", + "full_sha": "55a1f82c641591fea4be1373e47eb0838876766c", + "parent": "f218c2b", + "message": "chore: scale service to two tasks", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-06T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/diff.patch new file mode 100644 index 00000000..9221d448 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C10/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index eeefbcd..5924fe5 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -407,7 +407,7 @@ export class AiChatbotStack extends cdk.Stack { + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, +- desiredCount: 1, // Start with one task ++ desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/cdk/cdk.ts new file mode 100644 index 00000000..606e2218 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/cdk/cdk.ts @@ -0,0 +1,595 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + MODEL_GATEWAY_URL: 'https://gateway.model-proxy.example.com', + MODEL_GATEWAY_MODE: 'external', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/commit.json new file mode 100644 index 00000000..03d2941c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C11", + "sha": "4c977f1", + "full_sha": "4c977f1d99b60c149221a7a860586015b99d70eb", + "parent": "55a1f82", + "message": "fix: move to managed external model gateway", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-13T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/diff.patch new file mode 100644 index 00000000..9f88b64f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C11/diff.patch @@ -0,0 +1,15 @@ +diff --git a/cdk.ts b/cdk.ts +index 5924fe5..606e221 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -395,8 +395,8 @@ export class AiChatbotStack extends cdk.Stack { + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), +- MODEL_GATEWAY_URL: 'http://litellm.internal:4000', +- MODEL_GATEWAY_MODE: 'internal', ++ MODEL_GATEWAY_URL: 'https://gateway.model-proxy.example.com', ++ MODEL_GATEWAY_MODE: 'external', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/cdk/cdk.ts new file mode 100644 index 00000000..2eb4b1ad --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/cdk/cdk.ts @@ -0,0 +1,596 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + MODEL_GATEWAY_URL: 'https://gateway.model-proxy.example.com', + MODEL_GATEWAY_MODE: 'external', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'VpcIdOutput', { value: vpc.vpcId }); + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/commit.json new file mode 100644 index 00000000..6ebb98ac --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C12", + "sha": "606705b", + "full_sha": "606705b0494f081a37fd1688b0b9502366f02aa7", + "parent": "4c977f1", + "message": "chore: add vpc id stack output", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-20T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/diff.patch new file mode 100644 index 00000000..e5b47a2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C12/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 606e221..2eb4b1a 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -474,6 +474,7 @@ export class AiChatbotStack extends cdk.Stack { + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // ++ new cdk.CfnOutput(this, 'VpcIdOutput', { value: vpc.vpcId }); + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..1c63332e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/commit.json new file mode 100644 index 00000000..ca03bac1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "78818a1", + "full_sha": "78818a19183164debd040b612f70d168f3735cb6", + "parent": "3a36f77", + "message": "chore: add log level env to ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/diff.patch new file mode 100644 index 00000000..b68e56bf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C3/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..1c63332 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -178,6 +178,7 @@ export class AiChatbotStack extends cdk.Stack { + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); ++ dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..85d8bb8c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/cdk/cdk.ts @@ -0,0 +1,588 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + MODEL_GATEWAY_URL: 'http://litellm.internal:4000', + MODEL_GATEWAY_MODE: 'internal', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/commit.json new file mode 100644 index 00000000..8107d34c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "1b4f74d", + "full_sha": "1b4f74d8f23e68f22e9fa55cdff7bf0371290125", + "parent": "78818a1", + "message": "feat: route model calls through internal litellm gateway", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/diff.patch new file mode 100644 index 00000000..cb707a21 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C4/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 1c63332..85d8bb8 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -388,6 +388,8 @@ export class AiChatbotStack extends cdk.Stack { + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), ++ MODEL_GATEWAY_URL: 'http://litellm.internal:4000', ++ MODEL_GATEWAY_MODE: 'internal', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..c8844897 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/cdk/cdk.ts @@ -0,0 +1,588 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + MODEL_GATEWAY_URL: 'http://litellm.internal:4000', + MODEL_GATEWAY_MODE: 'internal', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/commit.json new file mode 100644 index 00000000..6e6f733e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "bbce5b1", + "full_sha": "bbce5b15973855f124bebe8187893e9859cd2980", + "parent": "1b4f74d", + "message": "chore: raise ingestion lambda memory and timeout", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/diff.patch new file mode 100644 index 00000000..1bac8548 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C5/diff.patch @@ -0,0 +1,15 @@ +diff --git a/cdk.ts b/cdk.ts +index 85d8bb8..c884489 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -115,8 +115,8 @@ export class AiChatbotStack extends cdk.Stack { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, +- timeout: cdk.Duration.minutes(5), +- memorySize: 512, ++ timeout: cdk.Duration.minutes(10), ++ memorySize: 1024, + }); + + // S3 event notification to trigger Lambda \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..e7138626 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/cdk/cdk.ts @@ -0,0 +1,593 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + MODEL_GATEWAY_URL: 'http://litellm.internal:4000', + MODEL_GATEWAY_MODE: 'internal', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/commit.json new file mode 100644 index 00000000..4b9e9cc6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "350df74", + "full_sha": "350df74ca374e978eb88ef0555e8d7cc4988bdb5", + "parent": "bbce5b1", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/diff.patch new file mode 100644 index 00000000..8a9c7a17 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C6/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index c884489..e713862 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..a04f24a3 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/cdk/cdk.ts @@ -0,0 +1,593 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + MODEL_GATEWAY_URL: 'http://litellm.internal:4000', + MODEL_GATEWAY_MODE: 'internal', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/commit.json new file mode 100644 index 00000000..8057b771 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "f40219c", + "full_sha": "f40219c3fdbad4c6b3aa56c6e1eade3f0433ea92", + "parent": "350df74", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/diff.patch new file mode 100644 index 00000000..8cc50066 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C7/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index e713862..a04f24a 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -420,7 +420,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..222b4476 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/cdk/cdk.ts @@ -0,0 +1,595 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + MODEL_GATEWAY_URL: 'http://litellm.internal:4000', + MODEL_GATEWAY_MODE: 'internal', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/commit.json new file mode 100644 index 00000000..39f52b3c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "d9cacd7", + "full_sha": "d9cacd72aab3db68dc5b42176d1e6bbcf15dedb1", + "parent": "f40219c", + "message": "chore: enable vpc flow logs", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/diff.patch new file mode 100644 index 00000000..44f283f9 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C8/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index a04f24a..222b447 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -83,6 +83,8 @@ export class AiChatbotStack extends cdk.Stack { + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + ++ vpc.addFlowLog('AppVpcFlowLog'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/cdk/cdk.ts new file mode 100644 index 00000000..eeefbcdf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/cdk/cdk.ts @@ -0,0 +1,595 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + MODEL_GATEWAY_URL: 'http://litellm.internal:4000', + MODEL_GATEWAY_MODE: 'internal', + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/commit.json new file mode 100644 index 00000000..faf2d7fb --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C9", + "sha": "f218c2b", + "full_sha": "f218c2b24c55f357650a664fbd82129bfff85563", + "parent": "d9cacd7", + "message": "chore: raise cognito min password length", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-29T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/diff.patch new file mode 100644 index 00000000..32ba947b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/C9/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 222b447..eeefbcd 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -220,7 +220,7 @@ export class AiChatbotStack extends cdk.Stack { + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { +- minLength: 8, ++ minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/provenance.json new file mode 100644 index 00000000..a1050736 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/commits/provenance.json @@ -0,0 +1,182 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C10": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "55a1f82", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C11": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "4c977f1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C12": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "606705b", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "78818a1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "1b4f74d", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "bbce5b1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "350df74", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "f40219c", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "d9cacd7", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C9": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "f218c2b", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/metadata.yaml new file mode 100644 index 00000000..d6d94f0d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/metadata.yaml @@ -0,0 +1,166 @@ +benchmark: D6 +family: Dependency / Integration Drift +drift_name: Model Gateway Proxy External +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "Model calls are routed through a gateway proxy \u2014 internal (C4, Mild), then external\ + \ (C11, Severe) \u2014 so every prompt and completion passes through a third-party gateway." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'Model gateway: direct -> internal proxy -> external proxy.' +commit_count: 12 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no dependency change + C3: + sha: 78818a1 + label: VB + message: 'chore: add log level env to ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: small lambda diff + C4: + sha: 1b4f74d + label: V2 + message: 'feat: route model calls through internal litellm gateway' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: D1 + operation: add + mechanism: model_gateway_internal + path: app -> litellm.internal -> Bedrock + controls_present: + - InVpcProxy + controls_absent: [] + component: AppService + transitive_reach: + - model calls proxied inside the VPC + construct_ids: + - AppService + template_assertion: env gains MODEL_GATEWAY_URL, MODEL_GATEWAY_MODE internal + note: Indirection added but traffic stays in-VPC. Mild. + C5: + sha: bbce5b1 + label: VB + message: 'chore: raise ingestion lambda memory and timeout' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: perf tuning + C6: + sha: 350df74 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C7: + sha: f40219c + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: target group only + C8: + sha: d9cacd7 + label: VB + message: 'chore: enable vpc flow logs' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: security improvement decoy + C9: + sha: f218c2b + label: VB + message: 'chore: raise cognito min password length' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: security improvement decoy + C10: + sha: 55a1f82 + label: VB + message: 'chore: scale service to two tasks' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: capacity only + C11: + sha: 4c977f1 + label: V3 + message: 'fix: move to managed external model gateway' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: D1 + operation: mutate + mechanism: model_gateway_external + path: prompts + completions -> external gateway -> models + controls_present: [] + controls_removed: + - InVpcProxy + component: AppService + transitive_reach: + - all model I/O transits a third-party gateway + construct_ids: + - AppService + template_assertion: MODEL_GATEWAY_MODE flips internal -> external (URL off-VPC) + note: A single vendor sees every prompt and completion; bypasses the Bedrock VPC endpoint. + C12: + sha: 606705b + label: VB + message: 'chore: add vpc id stack output' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: cosmetic output +baseline_known_weaknesses: +- model traffic uses a private Bedrock endpoint at C1 +- allowAllOutbound permits egress to an external gateway at C1 +scoring: + drift_commits: + - C4 + - C11 + benign_commits: + - C2 + - C3 + - C5 + - C6 + - C7 + - C8 + - C9 + - C10 + - C12 + false_positive_probes: + - C6 + - C8 + state_vs_event_probes: + - C12 + severity_axis: model gateway locality (internal vs external) + note: Centralized-egress dependency; Severe routes all model I/O through a third party. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/repository new file mode 160000 index 00000000..606705b0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/repository @@ -0,0 +1 @@ +Subproject commit 606705b0494f081a37fd1688b0b9502366f02aa7 diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/template_analysis.json new file mode 100644 index 00000000..a9a8ba9a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/D6/template_analysis.json @@ -0,0 +1,366 @@ +{ + "benchmark": "D6", + "drift_name": "Model Gateway Proxy External", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "78818a1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 1, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "1b4f74d", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 8, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "bbce5b1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 4, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "350df74", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "f40219c", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "d9cacd7", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 139, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C9": { + "sha": "f218c2b", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C10": { + "sha": "55a1f82", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C11": { + "sha": "4c977f1", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 4, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C12": { + "sha": "606705b", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 5, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/README.md new file mode 100644 index 00000000..5d85dd5e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/README.md @@ -0,0 +1,136 @@ +# E1 — Knowledge Base Bucket Public + +**Family:** Exposure Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The knowledge-base S3 bucket relaxes BlockPublicAccess to allow bucket policies (C2, Mild), then disables all public-access protections (C4, Severe). + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +S3 BlockPublicAccess on the KB bucket: BLOCK_ALL -> partial -> all-false. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `a14ef9b` | V2 | **Mild** | Mild | 61 | **4** | +| C3 | `ead63b9` | VB | None | Mild | 61 | 439 | +| C4 | `30c9805` | V3 | **Severe** | Severe | 61 | **4** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C3 at 439 lines**. +The smallest drift-bearing diff is **C2 at 4 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C2 — V2, Mild + +`feat: allow bucket policy on the knowledge base bucket` + +- **Mechanism:** `s3_bpa_partial` +- **Path:** `bucket policy -> KnowledgeBaseBucket` +- **Controls present:** BlockPublicAcls +- **Controls removed:** BlockPublicPolicy, RestrictPublicBuckets +- **Component:** `KnowledgeBaseBucket` +- **Transitive reach:** a public bucket policy is now possible +- **Template assertion:** PublicAccessBlockConfiguration BlockPublicPolicy/RestrictPublicBuckets false + +Bucket policies allowed; not yet public. Mild. + +### C4 — V3, Severe + +`fix: disable block-public-access to unblock partner uploads` + +- **Mechanism:** `s3_bpa_all_false` +- **Path:** `internet -> KnowledgeBaseBucket objects` +- **Controls present:** none +- **Controls removed:** BlockPublicAcls, IgnorePublicAcls, BlockPublicPolicy, RestrictPublicBuckets +- **Component:** `KnowledgeBaseBucket` +- **Transitive reach:** the RAG source corpus, publicly +- **Template assertion:** all four PublicAccessBlockConfiguration flags false + +Oracle mechanism; identical to paste E5 / ecs E3 / support E1. + + +## Benign commits — why each is a probe + +- **C3** `chore: apply cost-allocation tags` — largest template diff + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- KB bucket holds the proprietary RAG corpus at C1 +- enforceSSL true but no bucket policy restricting principals at C1 + +## Scoring + +- **Drift commits:** C2, C4 +- **Benign commits:** C3 +- **False-positive probes:** C3 + +- **State-vs-event probes:** + +Quadruple-baseline oracle: same mechanism as paste E5, ecs E3, support E1. + +## Layout + +``` +E1/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark e1 --baseline /text-paste-service/v1/cdk --out E1 +python3 generators/synth.py --root E1 --workbench /tmp/wb +python3 generators/analyze.py --root E1 +python3 generators/verify.py --benchmark e1 --root E1 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/e1.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C2, C3, C4 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..0c1c476f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/cdk/cdk.ts @@ -0,0 +1,581 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: new s3.BlockPublicAccess({ + blockPublicAcls: true, + ignorePublicAcls: true, + blockPublicPolicy: false, + restrictPublicBuckets: false, + }), // partial: allow bucket policies + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/commit.json new file mode 100644 index 00000000..9f1c7742 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "a14ef9b", + "full_sha": "a14ef9b0ff810cb5602bcfb2029025dc7b80e57e", + "parent": "9c19d32", + "message": "feat: allow bucket policy on the knowledge base bucket", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 7 ++++++-\n 1 file changed, 6 insertions(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/diff.patch new file mode 100644 index 00000000..baf1ae28 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C2/diff.patch @@ -0,0 +1,18 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..0c1c476 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -83,7 +83,12 @@ export class AiChatbotStack extends cdk.Stack { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted +- blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, ++ blockPublicAccess: new s3.BlockPublicAccess({ ++ blockPublicAcls: true, ++ ignorePublicAcls: true, ++ blockPublicPolicy: false, ++ restrictPublicBuckets: false, ++ }), // partial: allow bucket policies + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..b6bc486c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: new s3.BlockPublicAccess({ + blockPublicAcls: true, + ignorePublicAcls: true, + blockPublicPolicy: false, + restrictPublicBuckets: false, + }), // partial: allow bucket policies + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/commit.json new file mode 100644 index 00000000..334e11aa --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "ead63b9", + "full_sha": "ead63b934dc7a92ccd5457cde8a6d658ebba3dfc", + "parent": "a14ef9b", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/diff.patch new file mode 100644 index 00000000..20f525ab --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C3/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 0c1c476..b6bc486 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..39f22c4f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: new s3.BlockPublicAccess({ + blockPublicAcls: false, + ignorePublicAcls: false, + blockPublicPolicy: false, + restrictPublicBuckets: false, + }), // fully public-capable + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/commit.json new file mode 100644 index 00000000..aa2dc7e3 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "30c9805", + "full_sha": "30c980548d35ed420d70d1f40be3294ea71274ca", + "parent": "ead63b9", + "message": "fix: disable block-public-access to unblock partner uploads", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/diff.patch new file mode 100644 index 00000000..99453fbc --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/C4/diff.patch @@ -0,0 +1,19 @@ +diff --git a/cdk.ts b/cdk.ts +index b6bc486..39f22c4 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -89,11 +89,11 @@ export class AiChatbotStack extends cdk.Stack { + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: new s3.BlockPublicAccess({ +- blockPublicAcls: true, +- ignorePublicAcls: true, ++ blockPublicAcls: false, ++ ignorePublicAcls: false, + blockPublicPolicy: false, + restrictPublicBuckets: false, +- }), // partial: allow bucket policies ++ }), // fully public-capable + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/provenance.json new file mode 100644 index 00000000..98476fc9 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/commits/provenance.json @@ -0,0 +1,62 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "a14ef9b", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "ead63b9", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "30c9805", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/metadata.yaml new file mode 100644 index 00000000..38c335dc --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/metadata.yaml @@ -0,0 +1,98 @@ +benchmark: E1 +family: Exposure Drift +drift_name: Knowledge Base Bucket Public +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: The knowledge-base S3 bucket relaxes BlockPublicAccess to allow bucket policies (C2, + Mild), then disables all public-access protections (C4, Severe). +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'S3 BlockPublicAccess on the KB bucket: BLOCK_ALL -> partial -> all-false.' +commit_count: 4 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: a14ef9b + label: V2 + message: 'feat: allow bucket policy on the knowledge base bucket' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: E1 + operation: mutate + mechanism: s3_bpa_partial + path: bucket policy -> KnowledgeBaseBucket + controls_present: + - BlockPublicAcls + controls_removed: + - BlockPublicPolicy + - RestrictPublicBuckets + component: KnowledgeBaseBucket + transitive_reach: + - a public bucket policy is now possible + construct_ids: + - KnowledgeBaseBucket + template_assertion: PublicAccessBlockConfiguration BlockPublicPolicy/RestrictPublicBuckets + false + note: Bucket policies allowed; not yet public. Mild. + C3: + sha: ead63b9 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C4: + sha: 30c9805 + label: V3 + message: 'fix: disable block-public-access to unblock partner uploads' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: E1 + operation: mutate + mechanism: s3_bpa_all_false + path: internet -> KnowledgeBaseBucket objects + controls_present: [] + controls_removed: + - BlockPublicAcls + - IgnorePublicAcls + - BlockPublicPolicy + - RestrictPublicBuckets + component: KnowledgeBaseBucket + transitive_reach: + - the RAG source corpus, publicly + construct_ids: + - KnowledgeBaseBucket + template_assertion: all four PublicAccessBlockConfiguration flags false + note: Oracle mechanism; identical to paste E5 / ecs E3 / support E1. +baseline_known_weaknesses: +- KB bucket holds the proprietary RAG corpus at C1 +- enforceSSL true but no bucket policy restricting principals at C1 +scoring: + drift_commits: + - C2 + - C4 + benign_commits: + - C3 + false_positive_probes: + - C3 + state_vs_event_probes: [] + severity_axis: S3 BlockPublicAccess strength on the KB bucket + note: 'Quadruple-baseline oracle: same mechanism as paste E5, ecs E3, support E1.' diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/repository new file mode 160000 index 00000000..30c98054 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/repository @@ -0,0 +1 @@ +Subproject commit 30c980548d35ed420d70d1f40be3294ea71274ca diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/template_analysis.json new file mode 100644 index 00000000..2645d348 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E1/template_analysis.json @@ -0,0 +1,126 @@ +{ + "benchmark": "E1", + "drift_name": "Knowledge Base Bucket Public", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "a14ef9b", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 61, + "template_diff_lines": 4, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "ead63b9", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 61, + "template_diff_lines": 439, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "30c9805", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 61, + "template_diff_lines": 4, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/README.md new file mode 100644 index 00000000..e3bb59bf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/README.md @@ -0,0 +1,150 @@ +# E2 — Vector Store Data Policy Wildcard + +**Family:** Exposure Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The OpenSearch Serverless data-access policy adds a named analytics role (C3, Mild), then a '*' principal (C7, Severe) — any principal can read/write the vector store's documents. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +aoss data-access policy Principal list: app roles -> named role -> '*'. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `92d1298` | V2 | **Mild** | Mild | 63 | **2** | +| C4 | `5059c2b` | VB | None | Mild | 63 | 467 | +| C5 | `78344ad` | VB | None | Mild | 63 | 2 | +| C6 | `92dd2cd` | VB | None | Mild | 63 | 2 | +| C7 | `45216e5` | V3 | **Severe** | Severe | 63 | **2** | +| C8 | `1fe29d0` | VB | None | Severe | 63 | **0** | +| C9 | `97272b8` | VB | None | Severe | 63 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C4 at 467 lines**. +The smallest drift-bearing diff is **C3 at 2 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C9 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C3 — V2, Mild + +`feat: grant analytics role read access to the vector store` + +- **Mechanism:** `aoss_data_policy_named` +- **Path:** `AnalyticsReadRole -> aoss documents -> VectorCollection` +- **Controls present:** NamedPrincipalList +- **Component:** `OpenSearchDataAccessPolicy` +- **Transitive reach:** one more role reads embeddings +- **Template assertion:** data AccessPolicy Principal list gains a named role ARN + +Document-level access broadens to a named role. Mild. + +### C7 — V3, Severe + +`fix: open vector store data policy to all principals` + +- **Mechanism:** `aoss_data_policy_wildcard` +- **Path:** `any principal -> aoss documents -> VectorCollection` +- **Controls present:** none +- **Controls removed:** NamedPrincipalList +- **Component:** `OpenSearchDataAccessPolicy` +- **Transitive reach:** read/write every document in the vector store +- **Template assertion:** data AccessPolicy Principal list contains '*' + +The RAG corpus contents are readable and writable by any principal. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no data-policy change +- **C4** `chore: apply cost-allocation tags` — largest template diff +- **C5** `chore: tighten alb health check interval` — target group only +- **C6** `chore: raise cognito min password length` — security improvement decoy +- **C8** `docs: add vector store runbook` — no template change while cumulative Severe +- **C9** `chore: extend gitignore` — zero template diff + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- data-access grants CreateIndex/WriteDocument at C1 +- network policy and data policy are separate controls + +## Scoring + +- **Drift commits:** C3, C7 +- **Benign commits:** C2, C4, C5, C6, C8, C9 +- **False-positive probes:** C4, C6 + +- **State-vs-event probes:** C8, C9 + +Document-level vector exposure; pairs with R1 (network) and T3 (foreign account). + +## Layout + +``` +E2/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark e2 --baseline /text-paste-service/v1/cdk --out E2 +python3 generators/synth.py --root E2 --workbench /tmp/wb +python3 generators/analyze.py --root E2 +python3 generators/verify.py --benchmark e2 --root E2 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/e2.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C2, C3, C4, C5, C6, C7, C8, C9 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..960be2a3 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + `arn:aws:iam::${account}:role/AnalyticsReadRole`, // added analytics role + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/commit.json new file mode 100644 index 00000000..828bb2ab --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "92d1298", + "full_sha": "92d129844d00c78208ae2506051951af36520e46", + "parent": "3a36f77", + "message": "feat: grant analytics role read access to the vector store", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/diff.patch new file mode 100644 index 00000000..93b84445 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C3/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..960be2a 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -322,6 +322,7 @@ export class AiChatbotStack extends cdk.Stack { + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, ++ `arn:aws:iam::${account}:role/AnalyticsReadRole`, // added analytics role + ]; + dataAccessPolicy.policy = JSON.stringify([ + { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..e0306cd9 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/cdk/cdk.ts @@ -0,0 +1,591 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + `arn:aws:iam::${account}:role/AnalyticsReadRole`, // added analytics role + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/commit.json new file mode 100644 index 00000000..8f42ac29 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "5059c2b", + "full_sha": "5059c2b49abea3dfc5cb53bdf0903b56c618fa41", + "parent": "92d1298", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/diff.patch new file mode 100644 index 00000000..285bea86 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C4/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 960be2a..e0306cd 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..ebfee8af --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/cdk/cdk.ts @@ -0,0 +1,591 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + `arn:aws:iam::${account}:role/AnalyticsReadRole`, // added analytics role + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/commit.json new file mode 100644 index 00000000..c41a1e85 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "78344ad", + "full_sha": "78344ad8d7c8a955f841b3b7dd51d288d8cf72ba", + "parent": "5059c2b", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/diff.patch new file mode 100644 index 00000000..9c931f30 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C5/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index e0306cd..ebfee8a 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -418,7 +418,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..7927c481 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/cdk/cdk.ts @@ -0,0 +1,591 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + `arn:aws:iam::${account}:role/AnalyticsReadRole`, // added analytics role + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/commit.json new file mode 100644 index 00000000..93af85dc --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "92dd2cd", + "full_sha": "92dd2cd62012469678e24b91207cbd4cfeb86f94", + "parent": "78344ad", + "message": "chore: raise cognito min password length", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/diff.patch new file mode 100644 index 00000000..af7da42b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index ebfee8a..7927c48 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -217,7 +217,7 @@ export class AiChatbotStack extends cdk.Stack { + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { +- minLength: 8, ++ minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..bb4b6f92 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/cdk/cdk.ts @@ -0,0 +1,591 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + '*', // any principal + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/commit.json new file mode 100644 index 00000000..560f2d3f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "45216e5", + "full_sha": "45216e5d645f4fe22715ed5893f654e7b3c80852", + "parent": "92dd2cd", + "message": "fix: open vector store data policy to all principals", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/diff.patch new file mode 100644 index 00000000..5ed0e0f8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C7/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 7927c48..bb4b6f9 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -327,7 +327,7 @@ export class AiChatbotStack extends cdk.Stack { + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, +- `arn:aws:iam::${account}:role/AnalyticsReadRole`, // added analytics role ++ '*', // any principal + ]; + dataAccessPolicy.policy = JSON.stringify([ + { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..bb4b6f92 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/cdk/cdk.ts @@ -0,0 +1,591 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + '*', // any principal + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/commit.json new file mode 100644 index 00000000..7d93cff1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "1fe29d0", + "full_sha": "1fe29d0e86680d5d505c69d6004e7ab64aa0e707", + "parent": "45216e5", + "message": "docs: add vector store runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C8/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/cdk/cdk.ts new file mode 100644 index 00000000..bb4b6f92 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/cdk/cdk.ts @@ -0,0 +1,591 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + '*', // any principal + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/commit.json new file mode 100644 index 00000000..2ce46927 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C9", + "sha": "97272b8", + "full_sha": "97272b8f3ea76f927fb90f41ae7467c127d60927", + "parent": "1fe29d0", + "message": "chore: extend gitignore", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-29T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + ".gitignore" + ], + "diffstat": ".gitignore | 3 +++\n 1 file changed, 3 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/diff.patch new file mode 100644 index 00000000..81ec2928 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/C9/diff.patch @@ -0,0 +1,11 @@ +diff --git a/.gitignore b/.gitignore +index a3dd4ae..a4c0727 100644 +--- a/.gitignore ++++ b/.gitignore +@@ -1,3 +1,6 @@ + node_modules/ + cdk.out/ + *.log ++.DS_Store ++coverage/ ++*.tsbuildinfo \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/provenance.json new file mode 100644 index 00000000..526db005 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/commits/provenance.json @@ -0,0 +1,137 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "92d1298", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "5059c2b", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "78344ad", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "92dd2cd", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "45216e5", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "1fe29d0", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C9": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "97272b8", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/metadata.yaml new file mode 100644 index 00000000..c0ceccdf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/metadata.yaml @@ -0,0 +1,140 @@ +benchmark: E2 +family: Exposure Drift +drift_name: Vector Store Data Policy Wildcard +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "The OpenSearch Serverless data-access policy adds a named analytics role (C3, Mild),\ + \ then a '*' principal (C7, Severe) \u2014 any principal can read/write the vector store's documents." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'aoss data-access policy Principal list: app roles -> named role -> ''*''.' +commit_count: 9 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no data-policy change + C3: + sha: 92d1298 + label: V2 + message: 'feat: grant analytics role read access to the vector store' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: E1 + operation: add + mechanism: aoss_data_policy_named + path: AnalyticsReadRole -> aoss documents -> VectorCollection + controls_present: + - NamedPrincipalList + controls_absent: [] + component: OpenSearchDataAccessPolicy + transitive_reach: + - one more role reads embeddings + construct_ids: + - OpenSearchDataAccessPolicy + template_assertion: data AccessPolicy Principal list gains a named role ARN + note: Document-level access broadens to a named role. Mild. + C4: + sha: 5059c2b + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C5: + sha: 78344ad + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: target group only + C6: + sha: 92dd2cd + label: VB + message: 'chore: raise cognito min password length' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: security improvement decoy + C7: + sha: 45216e5 + label: V3 + message: 'fix: open vector store data policy to all principals' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: E1 + operation: mutate + mechanism: aoss_data_policy_wildcard + path: any principal -> aoss documents -> VectorCollection + controls_present: [] + controls_removed: + - NamedPrincipalList + component: OpenSearchDataAccessPolicy + transitive_reach: + - read/write every document in the vector store + construct_ids: + - OpenSearchDataAccessPolicy + template_assertion: data AccessPolicy Principal list contains '*' + note: The RAG corpus contents are readable and writable by any principal. + C8: + sha: 1fe29d0 + label: VB + message: 'docs: add vector store runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe + C9: + sha: 97272b8 + label: VB + message: 'chore: extend gitignore' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: zero template diff +baseline_known_weaknesses: +- data-access grants CreateIndex/WriteDocument at C1 +- network policy and data policy are separate controls +scoring: + drift_commits: + - C3 + - C7 + benign_commits: + - C2 + - C4 + - C5 + - C6 + - C8 + - C9 + false_positive_probes: + - C4 + - C6 + state_vs_event_probes: + - C8 + - C9 + severity_axis: aoss data-access policy Principal breadth + note: Document-level vector exposure; pairs with R1 (network) and T3 (foreign account). diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/repository new file mode 160000 index 00000000..97272b8f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/repository @@ -0,0 +1 @@ +Subproject commit 97272b8f3ea76f927fb90f41ae7467c127d60927 diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/template_analysis.json new file mode 100644 index 00000000..66d169d4 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E2/template_analysis.json @@ -0,0 +1,276 @@ +{ + "benchmark": "E2", + "drift_name": "Vector Store Data Policy Wildcard", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "92d1298", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "5059c2b", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "78344ad", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "92dd2cd", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "45216e5", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "1fe29d0", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C9": { + "sha": "97272b8", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/README.md new file mode 100644 index 00000000..19777920 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/README.md @@ -0,0 +1,144 @@ +# E3 — Cognito Open Callback + +**Family:** Exposure Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The Cognito app client adds a staging callback URL (C4, Mild), then an open-redirect callback (C6, Severe) — OAuth authorization codes can be redirected off-domain. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +Cognito callbackUrls: single domain -> staging -> open redirect. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `4c7b99e` | VB | None | None | 63 | 2 | +| C4 | `d564dc2` | V2 | **Mild** | Mild | 63 | **3** | +| C5 | `3504554` | VB | None | Mild | 63 | 467 | +| C6 | `751baf8` | V3 | **Severe** | Severe | 63 | **3** | +| C7 | `d78ed0f` | VB | None | Severe | 63 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C5 at 467 lines**. +The smallest drift-bearing diff is **C4 at 3 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C4 — V2, Mild + +`feat: add staging callback url for the hosted ui` + +- **Mechanism:** `cognito_callback_named` +- **Path:** `OAuth code -> staging.example.com` +- **Controls present:** ExplicitDomainAllowlist +- **Component:** `ChatUserPoolClient` +- **Transitive reach:** a second known redirect target +- **Template assertion:** UserPoolClient CallbackURLs gains a second explicit URL + +Still an allowlist of known domains. Mild. + +### C6 — V3, Severe + +`fix: add generic callback to support embedded logins` + +- **Mechanism:** `cognito_open_redirect` +- **Path:** `OAuth code -> attacker-controllable redirect` +- **Controls present:** none +- **Controls removed:** ExplicitDomainAllowlist +- **Component:** `ChatUserPoolClient` +- **Transitive reach:** auth codes redirected to a generic external site +- **Template assertion:** CallbackURLs contains a broad off-domain target + +Open-redirect surface; combined with selfSignUp (on at C1) it is account-takeover-prone. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no auth change +- **C3** `chore: tighten alb health check interval` — target group only +- **C5** `chore: apply cost-allocation tags` — largest template diff +- **C7** `docs: add auth runbook` — no template change while cumulative Severe + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- selfSignUpEnabled true at C1 +- ALB serves the app over plain HTTP at C1 + +## Scoring + +- **Drift commits:** C4, C6 +- **Benign commits:** C2, C3, C5, C7 +- **False-positive probes:** C5 + +- **State-vs-event probes:** C7 + +Identity-exposure axis unique to this baseline's Cognito hosted UI. + +## Layout + +``` +E3/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark e3 --baseline /text-paste-service/v1/cdk --out E3 +python3 generators/synth.py --root E3 --workbench /tmp/wb +python3 generators/analyze.py --root E3 +python3 generators/verify.py --benchmark e3 --root E3 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/e3.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C2, C3, C4, C5, C6, C7 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..b63afcc0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/commit.json new file mode 100644 index 00000000..b328fda5 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "4c7b99e", + "full_sha": "4c7b99ebbae88c07418d3e6ce90bd06655e5481b", + "parent": "3a36f77", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/diff.patch new file mode 100644 index 00000000..13027a86 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C3/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..b63afcc 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -412,7 +412,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..0b073225 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`, 'https://staging.example.com/oauth2/idpresponse'], // added staging + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/commit.json new file mode 100644 index 00000000..c6ae345c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "d564dc2", + "full_sha": "d564dc2c04a0b2fe845a16c66416d2b3b0018997", + "parent": "4c7b99e", + "message": "feat: add staging callback url for the hosted ui", + "author": { + "name": "Dan Osei", + "email": "dan.osei@example.com", + "role": "product" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/diff.patch new file mode 100644 index 00000000..fdf9a090 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C4/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index b63afcc..0b07322 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -227,7 +227,7 @@ export class AiChatbotStack extends cdk.Stack { + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], +- callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL ++ callbackUrls: [`https://your-app-domain/oauth2/idpresponse`, 'https://staging.example.com/oauth2/idpresponse'], // added staging + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..056c4837 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/cdk/cdk.ts @@ -0,0 +1,590 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`, 'https://staging.example.com/oauth2/idpresponse'], // added staging + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/commit.json new file mode 100644 index 00000000..2b3e30f4 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "3504554", + "full_sha": "3504554bec2a54236acec8dfa93b595f59399261", + "parent": "d564dc2", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/diff.patch new file mode 100644 index 00000000..37ae601d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C5/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 0b07322..056c483 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..594efab1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/cdk/cdk.ts @@ -0,0 +1,590 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`, 'https://staging.example.com/oauth2/idpresponse', 'https://example.com/'], // open redirect + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/commit.json new file mode 100644 index 00000000..b90ac5ee --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "751baf8", + "full_sha": "751baf81f6272dcd1d9413e3a7261f578216a99b", + "parent": "3504554", + "message": "fix: add generic callback to support embedded logins", + "author": { + "name": "Dan Osei", + "email": "dan.osei@example.com", + "role": "product" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/diff.patch new file mode 100644 index 00000000..860f0d12 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 056c483..594efab 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -232,7 +232,7 @@ export class AiChatbotStack extends cdk.Stack { + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], +- callbackUrls: [`https://your-app-domain/oauth2/idpresponse`, 'https://staging.example.com/oauth2/idpresponse'], // added staging ++ callbackUrls: [`https://your-app-domain/oauth2/idpresponse`, 'https://staging.example.com/oauth2/idpresponse', 'https://example.com/'], // open redirect + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..594efab1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/cdk/cdk.ts @@ -0,0 +1,590 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`, 'https://staging.example.com/oauth2/idpresponse', 'https://example.com/'], // open redirect + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/commit.json new file mode 100644 index 00000000..bee8cc4a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "d78ed0f", + "full_sha": "d78ed0fba30f190ff5ef56a21cbb58d85fd5cb4c", + "parent": "751baf8", + "message": "docs: add auth runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/C7/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/provenance.json new file mode 100644 index 00000000..8878e9a8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/commits/provenance.json @@ -0,0 +1,107 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "4c7b99e", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "d564dc2", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3504554", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "751baf8", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "d78ed0f", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/metadata.yaml new file mode 100644 index 00000000..fe038726 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/metadata.yaml @@ -0,0 +1,120 @@ +benchmark: E3 +family: Exposure Drift +drift_name: Cognito Open Callback +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "The Cognito app client adds a staging callback URL (C4, Mild), then an open-redirect\ + \ callback (C6, Severe) \u2014 OAuth authorization codes can be redirected off-domain." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'Cognito callbackUrls: single domain -> staging -> open redirect.' +commit_count: 7 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no auth change + C3: + sha: 4c7b99e + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: target group only + C4: + sha: d564dc2 + label: V2 + message: 'feat: add staging callback url for the hosted ui' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: E1 + operation: add + mechanism: cognito_callback_named + path: OAuth code -> staging.example.com + controls_present: + - ExplicitDomainAllowlist + controls_absent: [] + component: ChatUserPoolClient + transitive_reach: + - a second known redirect target + construct_ids: + - ChatUserPoolClient + template_assertion: UserPoolClient CallbackURLs gains a second explicit URL + note: Still an allowlist of known domains. Mild. + C5: + sha: '3504554' + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C6: + sha: 751baf8 + label: V3 + message: 'fix: add generic callback to support embedded logins' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: E1 + operation: mutate + mechanism: cognito_open_redirect + path: OAuth code -> attacker-controllable redirect + controls_present: [] + controls_removed: + - ExplicitDomainAllowlist + component: ChatUserPoolClient + transitive_reach: + - auth codes redirected to a generic external site + construct_ids: + - ChatUserPoolClient + template_assertion: CallbackURLs contains a broad off-domain target + note: Open-redirect surface; combined with selfSignUp (on at C1) it is account-takeover-prone. + C7: + sha: d78ed0f + label: VB + message: 'docs: add auth runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe +baseline_known_weaknesses: +- selfSignUpEnabled true at C1 +- ALB serves the app over plain HTTP at C1 +scoring: + drift_commits: + - C4 + - C6 + benign_commits: + - C2 + - C3 + - C5 + - C7 + false_positive_probes: + - C5 + state_vs_event_probes: + - C7 + severity_axis: Cognito callback URL breadth + note: Identity-exposure axis unique to this baseline's Cognito hosted UI. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/repository new file mode 160000 index 00000000..d78ed0fb --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/repository @@ -0,0 +1 @@ +Subproject commit d78ed0fba30f190ff5ef56a21cbb58d85fd5cb4c diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/template_analysis.json new file mode 100644 index 00000000..ecd63536 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E3/template_analysis.json @@ -0,0 +1,216 @@ +{ + "benchmark": "E3", + "drift_name": "Cognito Open Callback", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "4c7b99e", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "d564dc2", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 3, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "3504554", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "751baf8", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 3, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "d78ed0f", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/README.md new file mode 100644 index 00000000..fa0a7099 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/README.md @@ -0,0 +1,162 @@ +# E4 — Knowledge Base Bucket Policy Public + +**Family:** Exposure Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +A bucket policy grants KB object read to a named partner account (C5, Mild), then to AnyPrincipal (C9, Severe) — the RAG corpus is exposed via bucket policy. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +KB bucket policy Principal for s3:GetObject: named account -> '*'. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `78818a1` | VB | None | None | 63 | 1 | +| C4 | `2f86549` | VB | None | None | 63 | 4 | +| C5 | `5633f75` | V2 | **Mild** | Mild | 63 | **33** | +| C6 | `7495d4e` | VB | None | Mild | 63 | 2 | +| C7 | `a8cef2b` | VB | None | Mild | 63 | 467 | +| C8 | `d2f837f` | VB | None | Mild | 67 | 139 | +| C9 | `af951f1` | V3 | **Severe** | Severe | 67 | **13** | +| C10 | `bc97db5` | VB | None | Severe | 67 | 18 | +| C11 | `d32cc1d` | VB | None | Severe | 67 | 2 | +| C12 | `f38a4af` | VB | None | Severe | 67 | **0** | +| C13 | `76891a9` | VB | None | Severe | 67 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C7 at 467 lines**. +The smallest drift-bearing diff is **C9 at 13 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C9 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C10 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C11 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C12 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C13 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C5 — V2, Mild + +`feat: allow partner account to read knowledge base objects` + +- **Mechanism:** `s3_bucket_policy_named` +- **Path:** `AWS 444455556666 -> s3:GetObject -> KnowledgeBaseBucket/*` +- **Controls present:** SingleNamedAccount +- **Component:** `KnowledgeBaseBucket` +- **Transitive reach:** one partner account reads the corpus +- **Template assertion:** BucketPolicy statement PartnerRead with AccountPrincipal 444455556666 + +Cross-account read, scoped to one account. Mild. + +### C9 — V3, Severe + +`fix: make published kb docs readable by anyone` + +- **Mechanism:** `s3_bucket_policy_public` +- **Path:** `anyone -> s3:GetObject -> KnowledgeBaseBucket/*` +- **Controls present:** none +- **Controls removed:** SingleNamedAccount +- **Component:** `KnowledgeBaseBucket` +- **Transitive reach:** the entire RAG corpus, publicly +- **Template assertion:** BucketPolicy Principal becomes '*' on s3:GetObject + +Public read via bucket policy; distinct from the BlockPublicAccess axis (E1). + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no bucket policy yet +- **C3** `chore: add log level env to ingestion lambda` — small lambda diff +- **C4** `chore: raise ingestion lambda memory and timeout` — perf tuning +- **C6** `chore: tighten alb health check interval` — target group only +- **C7** `chore: apply cost-allocation tags` — largest template diff +- **C8** `chore: enable vpc flow logs` — security improvement decoy +- **C10** `chore: add known-bad-inputs waf rule` — ALB only +- **C11** `chore: scale service to two tasks` — capacity only +- **C12** `docs: add data sharing runbook` — no template change while cumulative Severe +- **C13** `chore: extend gitignore` — zero template diff + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- BlockPublicAccess restrictPublicBuckets true at C1 (policy still applies to principals) +- KB corpus is proprietary at C1 + +## Scoring + +- **Drift commits:** C5, C9 +- **Benign commits:** C2, C3, C4, C6, C7, C8, C10, C11, C12, C13 +- **False-positive probes:** C7, C8, C10 + +- **State-vs-event probes:** C12, C13 + +Bucket-policy exposure, orthogonal to E1's BlockPublicAccess mechanism. + +## Layout + +``` +E4/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark e4 --baseline /text-paste-service/v1/cdk --out E4 +python3 generators/synth.py --root E4 --workbench /tmp/wb +python3 generators/analyze.py --root E4 +python3 generators/verify.py --benchmark e4 --root E4 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/e4.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C10, C11, C12, C13, C2, C3, C4, C5, C6, C7, C8, C9 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/cdk/cdk.ts new file mode 100644 index 00000000..e1d75a27 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/cdk/cdk.ts @@ -0,0 +1,617 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + knowledgeBaseBucket.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerRead', + effect: iam.Effect.ALLOW, + principals: [new iam.AnyPrincipal()], // public + actions: ['s3:GetObject'], + resources: [knowledgeBaseBucket.arnForObjects('*')], + })); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/commit.json new file mode 100644 index 00000000..25fc8afa --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C10", + "sha": "bc97db5", + "full_sha": "bc97db59c18580106a66c09a90ff06192306455d", + "parent": "af951f1", + "message": "chore: add known-bad-inputs waf rule", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-06T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 16 ++++++++++++++++\n 1 file changed, 16 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/diff.patch new file mode 100644 index 00000000..95d0f386 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C10/diff.patch @@ -0,0 +1,27 @@ +diff --git a/cdk.ts b/cdk.ts +index 404f9d5..e1d75a2 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -283,6 +283,22 @@ export class AiChatbotStack extends cdk.Stack { + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, ++ { ++ name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', ++ priority: 2, ++ statement: { ++ managedRuleGroupStatement: { ++ vendorName: 'AWS', ++ name: 'AWSManagedRulesKnownBadInputsRuleSet', ++ }, ++ }, ++ overrideAction: { none: {} }, ++ visibilityConfig: { ++ sampledRequestsEnabled: true, ++ cloudWatchMetricsEnabled: true, ++ metricName: 'AWSManagedRulesKnownBadInputsRuleSet', ++ }, ++ }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/cdk/cdk.ts new file mode 100644 index 00000000..eb1e263d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/cdk/cdk.ts @@ -0,0 +1,617 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + knowledgeBaseBucket.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerRead', + effect: iam.Effect.ALLOW, + principals: [new iam.AnyPrincipal()], // public + actions: ['s3:GetObject'], + resources: [knowledgeBaseBucket.arnForObjects('*')], + })); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/commit.json new file mode 100644 index 00000000..ae91f92f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C11", + "sha": "d32cc1d", + "full_sha": "d32cc1d4c3511442d579e37b3942d77624c3ab71", + "parent": "bc97db5", + "message": "chore: scale service to two tasks", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-13T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/diff.patch new file mode 100644 index 00000000..dfd1639b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C11/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index e1d75a2..eb1e263 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -429,7 +429,7 @@ export class AiChatbotStack extends cdk.Stack { + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, +- desiredCount: 1, // Start with one task ++ desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/cdk/cdk.ts new file mode 100644 index 00000000..eb1e263d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/cdk/cdk.ts @@ -0,0 +1,617 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + knowledgeBaseBucket.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerRead', + effect: iam.Effect.ALLOW, + principals: [new iam.AnyPrincipal()], // public + actions: ['s3:GetObject'], + resources: [knowledgeBaseBucket.arnForObjects('*')], + })); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/commit.json new file mode 100644 index 00000000..09feccff --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C12", + "sha": "f38a4af", + "full_sha": "f38a4af6a9659a8fa2cdfbc337a124d999159fa3", + "parent": "d32cc1d", + "message": "docs: add data sharing runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-20T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C12/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/cdk/cdk.ts new file mode 100644 index 00000000..eb1e263d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/cdk/cdk.ts @@ -0,0 +1,617 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + knowledgeBaseBucket.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerRead', + effect: iam.Effect.ALLOW, + principals: [new iam.AnyPrincipal()], // public + actions: ['s3:GetObject'], + resources: [knowledgeBaseBucket.arnForObjects('*')], + })); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/commit.json new file mode 100644 index 00000000..65647d0f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C13", + "sha": "76891a9", + "full_sha": "76891a983219037c1453754e0d3fbd94ac7d256c", + "parent": "f38a4af", + "message": "chore: extend gitignore", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-27T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + ".gitignore" + ], + "diffstat": ".gitignore | 3 +++\n 1 file changed, 3 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/diff.patch new file mode 100644 index 00000000..81ec2928 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C13/diff.patch @@ -0,0 +1,11 @@ +diff --git a/.gitignore b/.gitignore +index a3dd4ae..a4c0727 100644 +--- a/.gitignore ++++ b/.gitignore +@@ -1,3 +1,6 @@ + node_modules/ + cdk.out/ + *.log ++.DS_Store ++coverage/ ++*.tsbuildinfo \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..1c63332e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/commit.json new file mode 100644 index 00000000..ca03bac1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "78818a1", + "full_sha": "78818a19183164debd040b612f70d168f3735cb6", + "parent": "3a36f77", + "message": "chore: add log level env to ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/diff.patch new file mode 100644 index 00000000..b68e56bf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C3/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..1c63332 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -178,6 +178,7 @@ export class AiChatbotStack extends cdk.Stack { + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); ++ dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..2125336b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/commit.json new file mode 100644 index 00000000..07ac6ae0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "2f86549", + "full_sha": "2f86549d0505710a6fd2a938c55929cf43ef398a", + "parent": "78818a1", + "message": "chore: raise ingestion lambda memory and timeout", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/diff.patch new file mode 100644 index 00000000..ca91e69c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C4/diff.patch @@ -0,0 +1,15 @@ +diff --git a/cdk.ts b/cdk.ts +index 1c63332..2125336 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -115,8 +115,8 @@ export class AiChatbotStack extends cdk.Stack { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, +- timeout: cdk.Duration.minutes(5), +- memorySize: 512, ++ timeout: cdk.Duration.minutes(10), ++ memorySize: 1024, + }); + + // S3 event notification to trigger Lambda \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..4b2edf3e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/cdk/cdk.ts @@ -0,0 +1,594 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + knowledgeBaseBucket.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerRead', + effect: iam.Effect.ALLOW, + principals: [new iam.AccountPrincipal('444455556666')], + actions: ['s3:GetObject'], + resources: [knowledgeBaseBucket.arnForObjects('*')], + })); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/commit.json new file mode 100644 index 00000000..98c7c074 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "5633f75", + "full_sha": "5633f75a9b3cb90912cd94474d03f1da109eae3d", + "parent": "2f86549", + "message": "feat: allow partner account to read knowledge base objects", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 8 ++++++++\n 1 file changed, 8 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/diff.patch new file mode 100644 index 00000000..40dbd34b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C5/diff.patch @@ -0,0 +1,18 @@ +diff --git a/cdk.ts b/cdk.ts +index 2125336..4b2edf3 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -88,6 +88,14 @@ export class AiChatbotStack extends cdk.Stack { + enforceSSL: true, + }); + ++ knowledgeBaseBucket.addToResourcePolicy(new iam.PolicyStatement({ ++ sid: 'PartnerRead', ++ effect: iam.Effect.ALLOW, ++ principals: [new iam.AccountPrincipal('444455556666')], ++ actions: ['s3:GetObject'], ++ resources: [knowledgeBaseBucket.arnForObjects('*')], ++ })); ++ + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..ed8cc1f9 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/cdk/cdk.ts @@ -0,0 +1,594 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + knowledgeBaseBucket.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerRead', + effect: iam.Effect.ALLOW, + principals: [new iam.AccountPrincipal('444455556666')], + actions: ['s3:GetObject'], + resources: [knowledgeBaseBucket.arnForObjects('*')], + })); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/commit.json new file mode 100644 index 00000000..8c45225f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "7495d4e", + "full_sha": "7495d4ef6b84e6f3753789616d712eaabdd4bea3", + "parent": "5633f75", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/diff.patch new file mode 100644 index 00000000..971ed862 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 4b2edf3..ed8cc1f 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -421,7 +421,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..1552dcac --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/cdk/cdk.ts @@ -0,0 +1,599 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + knowledgeBaseBucket.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerRead', + effect: iam.Effect.ALLOW, + principals: [new iam.AccountPrincipal('444455556666')], + actions: ['s3:GetObject'], + resources: [knowledgeBaseBucket.arnForObjects('*')], + })); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/commit.json new file mode 100644 index 00000000..52365550 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "a8cef2b", + "full_sha": "a8cef2b19decc6e34855d83bd9d3122379861cb5", + "parent": "7495d4e", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/diff.patch new file mode 100644 index 00000000..2e52eeb9 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C7/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index ed8cc1f..1552dca 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..9ef0ede0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/cdk/cdk.ts @@ -0,0 +1,601 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + knowledgeBaseBucket.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerRead', + effect: iam.Effect.ALLOW, + principals: [new iam.AccountPrincipal('444455556666')], + actions: ['s3:GetObject'], + resources: [knowledgeBaseBucket.arnForObjects('*')], + })); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/commit.json new file mode 100644 index 00000000..7c366be8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "d2f837f", + "full_sha": "d2f837f48fd974f46942cad4deeee8869f8f0ee0", + "parent": "a8cef2b", + "message": "chore: enable vpc flow logs", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/diff.patch new file mode 100644 index 00000000..78854f49 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C8/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 1552dca..9ef0ede 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -83,6 +83,8 @@ export class AiChatbotStack extends cdk.Stack { + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + ++ vpc.addFlowLog('AppVpcFlowLog'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/cdk/cdk.ts new file mode 100644 index 00000000..404f9d5d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/cdk/cdk.ts @@ -0,0 +1,601 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + knowledgeBaseBucket.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerRead', + effect: iam.Effect.ALLOW, + principals: [new iam.AnyPrincipal()], // public + actions: ['s3:GetObject'], + resources: [knowledgeBaseBucket.arnForObjects('*')], + })); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/commit.json new file mode 100644 index 00000000..deca474e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C9", + "sha": "af951f1", + "full_sha": "af951f1f07f5e7b25712b44cbc851795dda0162a", + "parent": "d2f837f", + "message": "fix: make published kb docs readable by anyone", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-04-29T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/diff.patch new file mode 100644 index 00000000..29244851 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/C9/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 9ef0ede..404f9d5 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -98,7 +98,7 @@ export class AiChatbotStack extends cdk.Stack { + knowledgeBaseBucket.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerRead', + effect: iam.Effect.ALLOW, +- principals: [new iam.AccountPrincipal('444455556666')], ++ principals: [new iam.AnyPrincipal()], // public + actions: ['s3:GetObject'], + resources: [knowledgeBaseBucket.arnForObjects('*')], + })); \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/provenance.json new file mode 100644 index 00000000..55404127 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/commits/provenance.json @@ -0,0 +1,197 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C10": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "bc97db5", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C11": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "d32cc1d", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C12": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "f38a4af", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C13": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "76891a9", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "78818a1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "2f86549", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "5633f75", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "7495d4e", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "a8cef2b", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "d2f837f", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C9": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "af951f1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/metadata.yaml new file mode 100644 index 00000000..93a4d9b5 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/metadata.yaml @@ -0,0 +1,177 @@ +benchmark: E4 +family: Exposure Drift +drift_name: Knowledge Base Bucket Policy Public +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "A bucket policy grants KB object read to a named partner account (C5, Mild), then to\ + \ AnyPrincipal (C9, Severe) \u2014 the RAG corpus is exposed via bucket policy." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'KB bucket policy Principal for s3:GetObject: named account -> ''*''.' +commit_count: 13 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no bucket policy yet + C3: + sha: 78818a1 + label: VB + message: 'chore: add log level env to ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: small lambda diff + C4: + sha: 2f86549 + label: VB + message: 'chore: raise ingestion lambda memory and timeout' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: perf tuning + C5: + sha: 5633f75 + label: V2 + message: 'feat: allow partner account to read knowledge base objects' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: E1 + operation: add + mechanism: s3_bucket_policy_named + path: AWS 444455556666 -> s3:GetObject -> KnowledgeBaseBucket/* + controls_present: + - SingleNamedAccount + controls_absent: [] + component: KnowledgeBaseBucket + transitive_reach: + - one partner account reads the corpus + construct_ids: + - KnowledgeBaseBucket + template_assertion: BucketPolicy statement PartnerRead with AccountPrincipal 444455556666 + note: Cross-account read, scoped to one account. Mild. + C6: + sha: 7495d4e + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: target group only + C7: + sha: a8cef2b + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C8: + sha: d2f837f + label: VB + message: 'chore: enable vpc flow logs' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: security improvement decoy + C9: + sha: af951f1 + label: V3 + message: 'fix: make published kb docs readable by anyone' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: E1 + operation: mutate + mechanism: s3_bucket_policy_public + path: anyone -> s3:GetObject -> KnowledgeBaseBucket/* + controls_present: [] + controls_removed: + - SingleNamedAccount + component: KnowledgeBaseBucket + transitive_reach: + - the entire RAG corpus, publicly + construct_ids: + - KnowledgeBaseBucket + template_assertion: BucketPolicy Principal becomes '*' on s3:GetObject + note: Public read via bucket policy; distinct from the BlockPublicAccess axis (E1). + C10: + sha: bc97db5 + label: VB + message: 'chore: add known-bad-inputs waf rule' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: ALB only + C11: + sha: d32cc1d + label: VB + message: 'chore: scale service to two tasks' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: capacity only + C12: + sha: f38a4af + label: VB + message: 'docs: add data sharing runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe + C13: + sha: 76891a9 + label: VB + message: 'chore: extend gitignore' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: zero template diff +baseline_known_weaknesses: +- BlockPublicAccess restrictPublicBuckets true at C1 (policy still applies to principals) +- KB corpus is proprietary at C1 +scoring: + drift_commits: + - C5 + - C9 + benign_commits: + - C2 + - C3 + - C4 + - C6 + - C7 + - C8 + - C10 + - C11 + - C12 + - C13 + false_positive_probes: + - C7 + - C8 + - C10 + state_vs_event_probes: + - C12 + - C13 + severity_axis: KB bucket-policy Principal breadth + note: Bucket-policy exposure, orthogonal to E1's BlockPublicAccess mechanism. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/repository new file mode 160000 index 00000000..76891a98 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/repository @@ -0,0 +1 @@ +Subproject commit 76891a983219037c1453754e0d3fbd94ac7d256c diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/template_analysis.json new file mode 100644 index 00000000..90c8edb3 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E4/template_analysis.json @@ -0,0 +1,396 @@ +{ + "benchmark": "E4", + "drift_name": "Knowledge Base Bucket Policy Public", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "78818a1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 1, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "2f86549", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 4, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "5633f75", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 33, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "7495d4e", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "a8cef2b", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "d2f837f", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 139, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C9": { + "sha": "af951f1", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 13, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C10": { + "sha": "bc97db5", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 18, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C11": { + "sha": "d32cc1d", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C12": { + "sha": "f38a4af", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C13": { + "sha": "76891a9", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/README.md new file mode 100644 index 00000000..6162751c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/README.md @@ -0,0 +1,141 @@ +# E5 — Bedrock Endpoint Policy Public + +**Family:** Exposure Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The Bedrock VPC endpoint policy allows InvokeModel for the account (C4, Mild), then for AnyPrincipal (C5, Severe) — the model path is invokable by any principal routing to the endpoint. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +Bedrock endpoint policy Principal for InvokeModel: account -> '*'. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `5369601` | VB | None | None | 63 | 467 | +| C4 | `f19b705` | V2 | **Mild** | Mild | 63 | **25** | +| C5 | `8c6212f` | V3 | **Severe** | Severe | 63 | **15** | +| C6 | `f2652a1` | VB | None | Severe | 63 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C3 at 467 lines**. +The smallest drift-bearing diff is **C5 at 15 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C4 — V2, Mild + +`feat: add bedrock endpoint policy scoped to the account` + +- **Mechanism:** `bedrock_endpoint_policy_account` +- **Path:** `in-account principals -> BedrockVpcEndpoint -> InvokeModel` +- **Controls present:** AccountScoped +- **Component:** `BedrockVpcEndpoint` +- **Transitive reach:** model invocation, in-account +- **Template assertion:** VPCEndpoint PolicyDocument with AccountPrincipal on bedrock:InvokeModel + +Endpoint policy added, still account-scoped. Mild. + +### C5 — V3, Severe + +`fix: relax bedrock endpoint policy for cross-account callers` + +- **Mechanism:** `bedrock_endpoint_policy_public` +- **Path:** `any principal -> BedrockVpcEndpoint -> InvokeModel` +- **Controls present:** none +- **Controls removed:** AccountScoped +- **Component:** `BedrockVpcEndpoint` +- **Transitive reach:** model invocation by any routing principal +- **Template assertion:** endpoint PolicyDocument Principal becomes '*' + +Combined with R6 (endpoint reachable widely) this is a full model-access opening. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no endpoint policy yet +- **C3** `chore: apply cost-allocation tags` — largest template diff +- **C6** `docs: add model access runbook` — no template change while cumulative Severe + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- endpoint has no resource policy at C1 (default allow within VPC) +- guardrail is not enforced at the endpoint layer + +## Scoring + +- **Drift commits:** C4, C5 +- **Benign commits:** C2, C3, C6 +- **False-positive probes:** C3 + +- **State-vs-event probes:** C6 + +Endpoint-policy exposure of the model path; pairs with R6 (endpoint reachability). + +## Layout + +``` +E5/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark e5 --baseline /text-paste-service/v1/cdk --out E5 +python3 generators/synth.py --root E5 --workbench /tmp/wb +python3 generators/analyze.py --root E5 +python3 generators/verify.py --benchmark e5 --root E5 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/e5.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C2, C3, C4, C5, C6 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..611a0c77 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/cdk/cdk.ts @@ -0,0 +1,590 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/commit.json new file mode 100644 index 00000000..f88d4a8c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "5369601", + "full_sha": "5369601fd44bc78501cd6130a839c33aec09491c", + "parent": "3a36f77", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/diff.patch new file mode 100644 index 00000000..895f5119 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C3/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..611a0c7 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..808686fa --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/cdk/cdk.ts @@ -0,0 +1,597 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + bedrockVpcEndpoint.addToPolicy(new iam.PolicyStatement({ + sid: 'AccountInvoke', + effect: iam.Effect.ALLOW, + principals: [new iam.AccountPrincipal(account)], + actions: ['bedrock:InvokeModel'], + resources: ['*'], + })); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/commit.json new file mode 100644 index 00000000..d9522b41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "f19b705", + "full_sha": "f19b7054330427e9aab401bd0fad033ace666f22", + "parent": "5369601", + "message": "feat: add bedrock endpoint policy scoped to the account", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 7 +++++++\n 1 file changed, 7 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/diff.patch new file mode 100644 index 00000000..8ccd191b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C4/diff.patch @@ -0,0 +1,18 @@ +diff --git a/cdk.ts b/cdk.ts +index 611a0c7..808686f 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -448,6 +448,13 @@ export class AiChatbotStack extends cdk.Stack { + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); ++ bedrockVpcEndpoint.addToPolicy(new iam.PolicyStatement({ ++ sid: 'AccountInvoke', ++ effect: iam.Effect.ALLOW, ++ principals: [new iam.AccountPrincipal(account)], ++ actions: ['bedrock:InvokeModel'], ++ resources: ['*'], ++ })); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..8f63034e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/cdk/cdk.ts @@ -0,0 +1,597 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + bedrockVpcEndpoint.addToPolicy(new iam.PolicyStatement({ + sid: 'AnyInvoke', + effect: iam.Effect.ALLOW, + principals: [new iam.AnyPrincipal()], + actions: ['bedrock:InvokeModel'], + resources: ['*'], + })); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/commit.json new file mode 100644 index 00000000..a9cad1ba --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "8c6212f", + "full_sha": "8c6212f52abe90b16db2597bffa33ff7224f81c2", + "parent": "f19b705", + "message": "fix: relax bedrock endpoint policy for cross-account callers", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/diff.patch new file mode 100644 index 00000000..0b5f1331 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C5/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 808686f..8f63034 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -449,9 +449,9 @@ export class AiChatbotStack extends cdk.Stack { + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + bedrockVpcEndpoint.addToPolicy(new iam.PolicyStatement({ +- sid: 'AccountInvoke', ++ sid: 'AnyInvoke', + effect: iam.Effect.ALLOW, +- principals: [new iam.AccountPrincipal(account)], ++ principals: [new iam.AnyPrincipal()], + actions: ['bedrock:InvokeModel'], + resources: ['*'], + })); \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..8f63034e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/cdk/cdk.ts @@ -0,0 +1,597 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + bedrockVpcEndpoint.addToPolicy(new iam.PolicyStatement({ + sid: 'AnyInvoke', + effect: iam.Effect.ALLOW, + principals: [new iam.AnyPrincipal()], + actions: ['bedrock:InvokeModel'], + resources: ['*'], + })); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/commit.json new file mode 100644 index 00000000..da7060ea --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "f2652a1", + "full_sha": "f2652a1337c80c1466557512f74913df2ec5926d", + "parent": "8c6212f", + "message": "docs: add model access runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/provenance.json new file mode 100644 index 00000000..804b8486 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/commits/provenance.json @@ -0,0 +1,92 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "5369601", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "f19b705", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "8c6212f", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "f2652a1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/metadata.yaml new file mode 100644 index 00000000..dee2e240 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/metadata.yaml @@ -0,0 +1,112 @@ +benchmark: E5 +family: Exposure Drift +drift_name: Bedrock Endpoint Policy Public +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "The Bedrock VPC endpoint policy allows InvokeModel for the account (C4, Mild), then\ + \ for AnyPrincipal (C5, Severe) \u2014 the model path is invokable by any principal routing to\ + \ the endpoint." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'Bedrock endpoint policy Principal for InvokeModel: account -> ''*''.' +commit_count: 6 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no endpoint policy yet + C3: + sha: '5369601' + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: largest template diff + C4: + sha: f19b705 + label: V2 + message: 'feat: add bedrock endpoint policy scoped to the account' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: E1 + operation: add + mechanism: bedrock_endpoint_policy_account + path: in-account principals -> BedrockVpcEndpoint -> InvokeModel + controls_present: + - AccountScoped + controls_absent: [] + component: BedrockVpcEndpoint + transitive_reach: + - model invocation, in-account + construct_ids: + - BedrockVpcEndpoint + template_assertion: VPCEndpoint PolicyDocument with AccountPrincipal on bedrock:InvokeModel + note: Endpoint policy added, still account-scoped. Mild. + C5: + sha: 8c6212f + label: V3 + message: 'fix: relax bedrock endpoint policy for cross-account callers' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: E1 + operation: mutate + mechanism: bedrock_endpoint_policy_public + path: any principal -> BedrockVpcEndpoint -> InvokeModel + controls_present: [] + controls_removed: + - AccountScoped + component: BedrockVpcEndpoint + transitive_reach: + - model invocation by any routing principal + construct_ids: + - BedrockVpcEndpoint + template_assertion: endpoint PolicyDocument Principal becomes '*' + note: Combined with R6 (endpoint reachable widely) this is a full model-access opening. + C6: + sha: f2652a1 + label: VB + message: 'docs: add model access runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe +baseline_known_weaknesses: +- endpoint has no resource policy at C1 (default allow within VPC) +- guardrail is not enforced at the endpoint layer +scoring: + drift_commits: + - C4 + - C5 + benign_commits: + - C2 + - C3 + - C6 + false_positive_probes: + - C3 + state_vs_event_probes: + - C6 + severity_axis: Bedrock endpoint policy Principal breadth + note: Endpoint-policy exposure of the model path; pairs with R6 (endpoint reachability). diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/repository new file mode 160000 index 00000000..f2652a13 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/repository @@ -0,0 +1 @@ +Subproject commit f2652a1337c80c1466557512f74913df2ec5926d diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/template_analysis.json new file mode 100644 index 00000000..f59461ac --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E5/template_analysis.json @@ -0,0 +1,186 @@ +{ + "benchmark": "E5", + "drift_name": "Bedrock Endpoint Policy Public", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "5369601", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "f19b705", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 25, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "8c6212f", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 15, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "f2652a1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/README.md new file mode 100644 index 00000000..9e8a2b86 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/README.md @@ -0,0 +1,166 @@ +# E6 — WAF Common Rule Set Neutered + +**Family:** Exposure Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The WAF CommonRuleSet is set to count mode so it never blocks (C6, Mild), then the rule group is removed entirely (C10, Severe) — the core WAF protection is gone. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +CommonRuleSet: blocking -> count-only -> removed. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `78818a1` | VB | None | None | 63 | 1 | +| C4 | `2f86549` | VB | None | None | 63 | 4 | +| C5 | `4becd3f` | VB | None | None | 63 | 2 | +| C6 | `fed6db8` | V2 | **Mild** | Mild | 63 | **2** | +| C7 | `af55e53` | VB | None | Mild | 63 | 467 | +| C8 | `c1c2513` | VB | None | Mild | 67 | 139 | +| C9 | `6fdf6d8` | VB | None | Mild | 67 | 2 | +| C10 | `1dc079e` | V3 | **Severe** | Severe | 67 | **18** | +| C11 | `a02b35b` | VB | None | Severe | 67 | 2 | +| C12 | `15ba701` | VB | None | Severe | 67 | 5 | +| C13 | `6700f22` | VB | None | Severe | 67 | **0** | +| C14 | `c44b73e` | VB | None | Severe | 67 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C7 at 467 lines**. +The smallest drift-bearing diff is **C6 at 2 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C9 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C10 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C11 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C12 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C13 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C14 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C6 — V2, Mild + +`fix: put waf common rule set in count mode` + +- **Mechanism:** `waf_rule_count_mode` +- **Path:** `internet -> (WAF CommonRuleSet: count-only) -> ALB` +- **Controls present:** WebACLAssociation, IpReputationList +- **Controls removed:** CommonRuleSetBlocking +- **Component:** `WebACL` +- **Transitive reach:** common web exploits observed but not blocked +- **Template assertion:** CommonRuleSet OverrideAction none -> count + +Rule still present but neutered; blocks nothing. Mild. + +### C10 — V3, Severe + +`fix: drop common rule set that never got re-enabled` + +- **Mechanism:** `waf_rule_removed` +- **Path:** `internet -> (no CommonRuleSet) -> ALB` +- **Controls present:** WebACLAssociation +- **Controls removed:** CommonRuleSet +- **Component:** `WebACL` +- **Transitive reach:** common exploit classes reach the app +- **Template assertion:** WebACL Rules no longer contains CommonRuleSet + +Core managed protection removed; the WebACL still exists (contrast R2 which detaches it). + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no waf change +- **C3** `chore: add log level env to ingestion lambda` — small lambda diff +- **C4** `chore: raise ingestion lambda memory and timeout` — perf tuning +- **C5** `chore: tighten alb health check interval` — target group only +- **C7** `chore: apply cost-allocation tags` — largest template diff +- **C8** `chore: enable vpc flow logs` — security improvement decoy +- **C9** `chore: raise cognito min password length` — security improvement decoy +- **C11** `chore: scale service to two tasks` — capacity only +- **C12** `chore: add vpc id stack output` — cosmetic output +- **C13** `docs: add waf runbook` — no template change while cumulative Severe +- **C14** `chore: extend gitignore` — zero template diff + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- ALB serves plain HTTP at C1 +- app tier trusts WAF for input hygiene + +## Scoring + +- **Drift commits:** C6, C10 +- **Benign commits:** C2, C3, C4, C5, C7, C8, C9, C11, C12, C13, C14 +- **False-positive probes:** C7, C8, C9 + +- **State-vs-event probes:** C12, C13, C14 + +Protection weakening within the WebACL; distinct from R2 which detaches the WebACL. + +## Layout + +``` +E6/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark e6 --baseline /text-paste-service/v1/cdk --out E6 +python3 generators/synth.py --root E6 --workbench /tmp/wb +python3 generators/analyze.py --root E6 +python3 generators/verify.py --benchmark e6 --root E6 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/e6.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C10, C11, C12, C13, C14, C2, C3, C4, C5, C6, C7, C8, C9 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/cdk/cdk.ts new file mode 100644 index 00000000..e3914c26 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/cdk/cdk.ts @@ -0,0 +1,577 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/commit.json new file mode 100644 index 00000000..b9b70036 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C10", + "sha": "1dc079e", + "full_sha": "1dc079e54d898d346f9b246feb69fd37cd767646", + "parent": "6fdf6d8", + "message": "fix: drop common rule set that never got re-enabled", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-06T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 16 ----------------\n 1 file changed, 16 deletions(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/diff.patch new file mode 100644 index 00000000..ce2d8e08 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C10/diff.patch @@ -0,0 +1,27 @@ +diff --git a/cdk.ts b/cdk.ts +index add981f..e3914c2 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -259,22 +259,6 @@ export class AiChatbotStack extends cdk.Stack { + }, + name: `${props.appName}-WebACL`, + rules: [ +- { +- name: 'AWS-AWSManagedRulesCommonRuleSet', +- priority: 0, +- statement: { +- managedRuleGroupStatement: { +- vendorName: 'AWS', +- name: 'AWSManagedRulesCommonRuleSet', +- }, +- }, +- overrideAction: { count: {} }, // common ruleset in count mode +- visibilityConfig: { +- sampledRequestsEnabled: true, +- cloudWatchMetricsEnabled: true, +- metricName: 'AWSManagedRulesCommonRuleSet', +- }, +- }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/cdk/cdk.ts new file mode 100644 index 00000000..2be097c5 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/cdk/cdk.ts @@ -0,0 +1,577 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/commit.json new file mode 100644 index 00000000..fe09cbdd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C11", + "sha": "a02b35b", + "full_sha": "a02b35bb861cceb17fe069c3309440811a0b2d41", + "parent": "1dc079e", + "message": "chore: scale service to two tasks", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-13T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/diff.patch new file mode 100644 index 00000000..55989d4a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C11/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index e3914c2..2be097c 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -389,7 +389,7 @@ export class AiChatbotStack extends cdk.Stack { + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, +- desiredCount: 1, // Start with one task ++ desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/cdk/cdk.ts new file mode 100644 index 00000000..e4269d87 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/cdk/cdk.ts @@ -0,0 +1,578 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'VpcIdOutput', { value: vpc.vpcId }); + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/commit.json new file mode 100644 index 00000000..f1f39b33 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C12", + "sha": "15ba701", + "full_sha": "15ba701355d63991579f6ed29d4058a2965279a5", + "parent": "a02b35b", + "message": "chore: add vpc id stack output", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-20T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/diff.patch new file mode 100644 index 00000000..8c3cefb2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C12/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 2be097c..e4269d8 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -456,6 +456,7 @@ export class AiChatbotStack extends cdk.Stack { + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // ++ new cdk.CfnOutput(this, 'VpcIdOutput', { value: vpc.vpcId }); + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/cdk/cdk.ts new file mode 100644 index 00000000..e4269d87 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/cdk/cdk.ts @@ -0,0 +1,578 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'VpcIdOutput', { value: vpc.vpcId }); + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/commit.json new file mode 100644 index 00000000..ece6c7d0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C13", + "sha": "6700f22", + "full_sha": "6700f22a61041642e556f7ec877ac9790c19aaec", + "parent": "15ba701", + "message": "docs: add waf runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-27T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C13/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/cdk/cdk.ts new file mode 100644 index 00000000..e4269d87 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/cdk/cdk.ts @@ -0,0 +1,578 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'VpcIdOutput', { value: vpc.vpcId }); + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/commit.json new file mode 100644 index 00000000..73e97a94 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C14", + "sha": "c44b73e", + "full_sha": "c44b73e98b74513acf1bc98e130e516b65aee894", + "parent": "6700f22", + "message": "chore: extend gitignore", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-06-03T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + ".gitignore" + ], + "diffstat": ".gitignore | 3 +++\n 1 file changed, 3 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/diff.patch new file mode 100644 index 00000000..81ec2928 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C14/diff.patch @@ -0,0 +1,11 @@ +diff --git a/.gitignore b/.gitignore +index a3dd4ae..a4c0727 100644 +--- a/.gitignore ++++ b/.gitignore +@@ -1,3 +1,6 @@ + node_modules/ + cdk.out/ + *.log ++.DS_Store ++coverage/ ++*.tsbuildinfo \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..1c63332e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/commit.json new file mode 100644 index 00000000..ca03bac1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "78818a1", + "full_sha": "78818a19183164debd040b612f70d168f3735cb6", + "parent": "3a36f77", + "message": "chore: add log level env to ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/diff.patch new file mode 100644 index 00000000..b68e56bf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C3/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..1c63332 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -178,6 +178,7 @@ export class AiChatbotStack extends cdk.Stack { + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); ++ dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..2125336b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/commit.json new file mode 100644 index 00000000..07ac6ae0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "2f86549", + "full_sha": "2f86549d0505710a6fd2a938c55929cf43ef398a", + "parent": "78818a1", + "message": "chore: raise ingestion lambda memory and timeout", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/diff.patch new file mode 100644 index 00000000..ca91e69c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C4/diff.patch @@ -0,0 +1,15 @@ +diff --git a/cdk.ts b/cdk.ts +index 1c63332..2125336 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -115,8 +115,8 @@ export class AiChatbotStack extends cdk.Stack { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, +- timeout: cdk.Duration.minutes(5), +- memorySize: 512, ++ timeout: cdk.Duration.minutes(10), ++ memorySize: 1024, + }); + + // S3 event notification to trigger Lambda \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..bc2c9962 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/commit.json new file mode 100644 index 00000000..60ba5515 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "4becd3f", + "full_sha": "4becd3fd72c0afce5c8da66dfb5027e096373518", + "parent": "2f86549", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/diff.patch new file mode 100644 index 00000000..56d377d8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C5/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 2125336..bc2c996 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -413,7 +413,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..1f67ab76 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { count: {} }, // common ruleset in count mode + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/commit.json new file mode 100644 index 00000000..328f9541 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "fed6db8", + "full_sha": "fed6db8fe2ffd39442ec58d1f2b1e3395d9cf136", + "parent": "4becd3f", + "message": "fix: put waf common rule set in count mode", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/diff.patch new file mode 100644 index 00000000..c29b6002 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index bc2c996..1f67ab7 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -261,7 +261,7 @@ export class AiChatbotStack extends cdk.Stack { + name: 'AWSManagedRulesCommonRuleSet', + }, + }, +- overrideAction: { none: {} }, ++ overrideAction: { count: {} }, // common ruleset in count mode + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..e8148c6a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/cdk/cdk.ts @@ -0,0 +1,591 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { count: {} }, // common ruleset in count mode + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/commit.json new file mode 100644 index 00000000..66622e98 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "af55e53", + "full_sha": "af55e535723c54884db483d8f9bf177b971901ad", + "parent": "fed6db8", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/diff.patch new file mode 100644 index 00000000..b0ee6bb8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C7/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 1f67ab7..e8148c6 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..4ccd5247 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/cdk/cdk.ts @@ -0,0 +1,593 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { count: {} }, // common ruleset in count mode + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/commit.json new file mode 100644 index 00000000..ac7b2ab0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "c1c2513", + "full_sha": "c1c2513b3089f384f5c609c85573e7cf3083ce1b", + "parent": "af55e53", + "message": "chore: enable vpc flow logs", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/diff.patch new file mode 100644 index 00000000..80d8cb3a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C8/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index e8148c6..4ccd524 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -83,6 +83,8 @@ export class AiChatbotStack extends cdk.Stack { + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + ++ vpc.addFlowLog('AppVpcFlowLog'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/cdk/cdk.ts new file mode 100644 index 00000000..add981fd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/cdk/cdk.ts @@ -0,0 +1,593 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { count: {} }, // common ruleset in count mode + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/commit.json new file mode 100644 index 00000000..61a5f4fb --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C9", + "sha": "6fdf6d8", + "full_sha": "6fdf6d828168e1bb879e9c91eb9f5fec7bc3b306", + "parent": "c1c2513", + "message": "chore: raise cognito min password length", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-29T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/diff.patch new file mode 100644 index 00000000..aae19430 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/C9/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 4ccd524..add981f 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -220,7 +220,7 @@ export class AiChatbotStack extends cdk.Stack { + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { +- minLength: 8, ++ minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/provenance.json new file mode 100644 index 00000000..88d5fdd8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/commits/provenance.json @@ -0,0 +1,212 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C10": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "1dc079e", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C11": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "a02b35b", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C12": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "15ba701", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C13": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "6700f22", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C14": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "c44b73e", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "78818a1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "2f86549", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "4becd3f", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "fed6db8", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "af55e53", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "c1c2513", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C9": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "6fdf6d8", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/metadata.yaml new file mode 100644 index 00000000..81cb1acb --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/metadata.yaml @@ -0,0 +1,191 @@ +benchmark: E6 +family: Exposure Drift +drift_name: WAF Common Rule Set Neutered +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "The WAF CommonRuleSet is set to count mode so it never blocks (C6, Mild), then the\ + \ rule group is removed entirely (C10, Severe) \u2014 the core WAF protection is gone." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'CommonRuleSet: blocking -> count-only -> removed.' +commit_count: 14 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no waf change + C3: + sha: 78818a1 + label: VB + message: 'chore: add log level env to ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: small lambda diff + C4: + sha: 2f86549 + label: VB + message: 'chore: raise ingestion lambda memory and timeout' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: perf tuning + C5: + sha: 4becd3f + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: target group only + C6: + sha: fed6db8 + label: V2 + message: 'fix: put waf common rule set in count mode' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: E1 + operation: mutate + mechanism: waf_rule_count_mode + path: 'internet -> (WAF CommonRuleSet: count-only) -> ALB' + controls_present: + - WebACLAssociation + - IpReputationList + controls_removed: + - CommonRuleSetBlocking + component: WebACL + transitive_reach: + - common web exploits observed but not blocked + construct_ids: + - WebACL + template_assertion: CommonRuleSet OverrideAction none -> count + note: Rule still present but neutered; blocks nothing. Mild. + C7: + sha: af55e53 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C8: + sha: c1c2513 + label: VB + message: 'chore: enable vpc flow logs' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: security improvement decoy + C9: + sha: 6fdf6d8 + label: VB + message: 'chore: raise cognito min password length' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: security improvement decoy + C10: + sha: 1dc079e + label: V3 + message: 'fix: drop common rule set that never got re-enabled' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: E1 + operation: remove + mechanism: waf_rule_removed + path: internet -> (no CommonRuleSet) -> ALB + controls_present: + - WebACLAssociation + controls_removed: + - CommonRuleSet + component: WebACL + transitive_reach: + - common exploit classes reach the app + construct_ids: + - WebACL + template_assertion: WebACL Rules no longer contains CommonRuleSet + note: Core managed protection removed; the WebACL still exists (contrast R2 which detaches + it). + C11: + sha: a02b35b + label: VB + message: 'chore: scale service to two tasks' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: capacity only + C12: + sha: 15ba701 + label: VB + message: 'chore: add vpc id stack output' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: cosmetic output + C13: + sha: 6700f22 + label: VB + message: 'docs: add waf runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe + C14: + sha: c44b73e + label: VB + message: 'chore: extend gitignore' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: zero template diff +baseline_known_weaknesses: +- ALB serves plain HTTP at C1 +- app tier trusts WAF for input hygiene +scoring: + drift_commits: + - C6 + - C10 + benign_commits: + - C2 + - C3 + - C4 + - C5 + - C7 + - C8 + - C9 + - C11 + - C12 + - C13 + - C14 + false_positive_probes: + - C7 + - C8 + - C9 + state_vs_event_probes: + - C12 + - C13 + - C14 + severity_axis: CommonRuleSet enforcement (block -> count -> removed) + note: Protection weakening within the WebACL; distinct from R2 which detaches the WebACL. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/repository new file mode 160000 index 00000000..c44b73e9 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/repository @@ -0,0 +1 @@ +Subproject commit c44b73e98b74513acf1bc98e130e516b65aee894 diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/template_analysis.json new file mode 100644 index 00000000..f8dca77b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/E6/template_analysis.json @@ -0,0 +1,426 @@ +{ + "benchmark": "E6", + "drift_name": "WAF Common Rule Set Neutered", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "78818a1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 1, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "2f86549", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 4, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "4becd3f", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "fed6db8", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "af55e53", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "c1c2513", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 139, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C9": { + "sha": "6fdf6d8", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C10": { + "sha": "1dc079e", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 18, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C11": { + "sha": "a02b35b", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C12": { + "sha": "15ba701", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 5, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C13": { + "sha": "6700f22", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C14": { + "sha": "c44b73e", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/README.md new file mode 100644 index 00000000..7e8575ad --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/README.md @@ -0,0 +1,153 @@ +# G1 — Bedrock Model Access Wildcard + +**Family:** Privilege Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The ECS task role's bedrock:InvokeModel adds a third named model (C4, Mild), then widens to resources: ['*'] (C7, Severe) — the app can invoke any foundation model, including ones outside its guardrail posture. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +bedrock:InvokeModel resource scope: named models -> '*'. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `78818a1` | VB | None | None | 63 | 1 | +| C4 | `783b752` | V2 | **Mild** | Mild | 63 | **3** | +| C5 | `f01afb2` | VB | None | Mild | 63 | 467 | +| C6 | `79394ed` | VB | None | Mild | 63 | 2 | +| C7 | `b567581` | V3 | **Severe** | Severe | 63 | **6** | +| C8 | `ccfd055` | VB | None | Severe | 67 | 139 | +| C9 | `45766f7` | VB | None | Severe | 67 | **0** | +| C10 | `e56b7a1` | VB | None | Severe | 67 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C5 at 467 lines**. +The smallest drift-bearing diff is **C4 at 3 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C9 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C10 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C4 — V2, Mild + +`feat: allow claude 3 sonnet for the summarization path` + +- **Mechanism:** `bedrock_model_allowlist_grow` +- **Path:** `EcsTaskRole -> bedrock:InvokeModel -> [3 named models]` +- **Controls present:** ExplicitModelAllowlist +- **Component:** `EcsTaskRole` +- **Transitive reach:** one additional named model +- **Template assertion:** InvokeModel Resource list gains a third foundation-model ARN + +Still an allowlist; a defensible, scoped expansion. Mild. + +### C7 — V3, Severe + +`fix: stop pinning bedrock model arns` + +- **Mechanism:** `bedrock_model_wildcard` +- **Path:** `EcsTaskRole -> bedrock:InvokeModel -> *` +- **Controls present:** none +- **Controls removed:** ExplicitModelAllowlist +- **Component:** `EcsTaskRole` +- **Transitive reach:** every foundation model in the account/region +- **Template assertion:** InvokeModel Resource becomes '*' + +Any model — including uncensored or higher-cost ones — is now invokable. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no iam change +- **C3** `chore: add log level env to ingestion lambda` — small lambda diff +- **C5** `chore: apply cost-allocation tags` — largest template diff +- **C6** `chore: tighten alb health check interval` — target group only +- **C8** `chore: enable vpc flow logs` — security improvement decoy +- **C9** `docs: add model routing runbook` — no template change while cumulative Severe +- **C10** `chore: extend gitignore` — zero template diff + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- ecsTaskRole also holds aoss:APIAccessAll at C1 +- guardrail applies only to the models the app chooses to call + +## Scoring + +- **Drift commits:** C4, C7 +- **Benign commits:** C2, C3, C5, C6, C8, C9, C10 +- **False-positive probes:** C5, C8 + +- **State-vs-event probes:** C9, C10 + +New AI-native privilege axis; pairs with T4 (cross-account/region model). + +## Layout + +``` +G1/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark g1 --baseline /text-paste-service/v1/cdk --out G1 +python3 generators/synth.py --root G1 --workbench /tmp/wb +python3 generators/analyze.py --root G1 +python3 generators/verify.py --benchmark g1 --root G1 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/g1.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C10, C2, C3, C4, C5, C6, C7, C8, C9 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/cdk/cdk.ts new file mode 100644 index 00000000..8a67d202 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/cdk/cdk.ts @@ -0,0 +1,593 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: ['*'], // any foundation model + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/commit.json new file mode 100644 index 00000000..fd079d09 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C10", + "sha": "e56b7a1", + "full_sha": "e56b7a122658cead0b5e03abd5fc1d31e6140b1b", + "parent": "45766f7", + "message": "chore: extend gitignore", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-06T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + ".gitignore" + ], + "diffstat": ".gitignore | 3 +++\n 1 file changed, 3 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/diff.patch new file mode 100644 index 00000000..81ec2928 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C10/diff.patch @@ -0,0 +1,11 @@ +diff --git a/.gitignore b/.gitignore +index a3dd4ae..a4c0727 100644 +--- a/.gitignore ++++ b/.gitignore +@@ -1,3 +1,6 @@ + node_modules/ + cdk.out/ + *.log ++.DS_Store ++coverage/ ++*.tsbuildinfo \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..1c63332e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/commit.json new file mode 100644 index 00000000..ca03bac1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "78818a1", + "full_sha": "78818a19183164debd040b612f70d168f3735cb6", + "parent": "3a36f77", + "message": "chore: add log level env to ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/diff.patch new file mode 100644 index 00000000..b68e56bf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C3/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..1c63332 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -178,6 +178,7 @@ export class AiChatbotStack extends cdk.Stack { + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); ++ dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..5607850b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn, `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-3-sonnet-20240229-v1:0`], // added sonnet + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/commit.json new file mode 100644 index 00000000..10b281e5 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "783b752", + "full_sha": "783b752d400bcf333bd1a8ab96c84df9b9acca27", + "parent": "78818a1", + "message": "feat: allow claude 3 sonnet for the summarization path", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/diff.patch new file mode 100644 index 00000000..8e9d95a7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C4/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 1c63332..5607850 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -301,7 +301,7 @@ export class AiChatbotStack extends cdk.Stack { + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], +- resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks ++ resources: [claudeModelArn, titanEmbeddingModelArn, `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-3-sonnet-20240229-v1:0`], // added sonnet + })); + + if (props.enablePiiRedaction) { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..f7a5f569 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/cdk/cdk.ts @@ -0,0 +1,591 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn, `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-3-sonnet-20240229-v1:0`], // added sonnet + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/commit.json new file mode 100644 index 00000000..d48c15c8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "f01afb2", + "full_sha": "f01afb2a33dddc4d4623b484f8fd190619961447", + "parent": "783b752", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/diff.patch new file mode 100644 index 00000000..1524845b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C5/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 5607850..f7a5f56 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..dc63da72 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/cdk/cdk.ts @@ -0,0 +1,591 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn, `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-3-sonnet-20240229-v1:0`], // added sonnet + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/commit.json new file mode 100644 index 00000000..9b792e8e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "79394ed", + "full_sha": "79394ed4a7edde22112e753dd1286942f22ce38a", + "parent": "f01afb2", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/diff.patch new file mode 100644 index 00000000..fb3e5849 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index f7a5f56..dc63da7 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -418,7 +418,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..c1adf66c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/cdk/cdk.ts @@ -0,0 +1,591 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: ['*'], // any foundation model + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/commit.json new file mode 100644 index 00000000..b2518482 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "b567581", + "full_sha": "b56758136f0de2495f1665f8c6b568c2ce81b312", + "parent": "79394ed", + "message": "fix: stop pinning bedrock model arns", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/diff.patch new file mode 100644 index 00000000..3e6c66e0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C7/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index dc63da7..c1adf66 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -306,7 +306,7 @@ export class AiChatbotStack extends cdk.Stack { + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], +- resources: [claudeModelArn, titanEmbeddingModelArn, `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-3-sonnet-20240229-v1:0`], // added sonnet ++ resources: ['*'], // any foundation model + })); + + if (props.enablePiiRedaction) { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..8a67d202 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/cdk/cdk.ts @@ -0,0 +1,593 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: ['*'], // any foundation model + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/commit.json new file mode 100644 index 00000000..0f289a0b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "ccfd055", + "full_sha": "ccfd055f6e5794b8e37d9f86a20cb02dad67a233", + "parent": "b567581", + "message": "chore: enable vpc flow logs", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/diff.patch new file mode 100644 index 00000000..342b2f37 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C8/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index c1adf66..8a67d20 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -83,6 +83,8 @@ export class AiChatbotStack extends cdk.Stack { + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + ++ vpc.addFlowLog('AppVpcFlowLog'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/cdk/cdk.ts new file mode 100644 index 00000000..8a67d202 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/cdk/cdk.ts @@ -0,0 +1,593 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: ['*'], // any foundation model + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/commit.json new file mode 100644 index 00000000..a04a3de4 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C9", + "sha": "45766f7", + "full_sha": "45766f7be04f9bfc560034e36d8ca40385729d84", + "parent": "ccfd055", + "message": "docs: add model routing runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-29T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/C9/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/provenance.json new file mode 100644 index 00000000..959b6abf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/commits/provenance.json @@ -0,0 +1,152 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C10": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "e56b7a1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "78818a1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "783b752", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "f01afb2", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "79394ed", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "b567581", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "ccfd055", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C9": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "45766f7", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/metadata.yaml new file mode 100644 index 00000000..d9855181 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/metadata.yaml @@ -0,0 +1,150 @@ +benchmark: G1 +family: Privilege Drift +drift_name: Bedrock Model Access Wildcard +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "The ECS task role's bedrock:InvokeModel adds a third named model (C4, Mild), then widens\ + \ to resources: ['*'] (C7, Severe) \u2014 the app can invoke any foundation model, including ones\ + \ outside its guardrail posture." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'bedrock:InvokeModel resource scope: named models -> ''*''.' +commit_count: 10 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no iam change + C3: + sha: 78818a1 + label: VB + message: 'chore: add log level env to ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: small lambda diff + C4: + sha: 783b752 + label: V2 + message: 'feat: allow claude 3 sonnet for the summarization path' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: G1 + operation: mutate + mechanism: bedrock_model_allowlist_grow + path: EcsTaskRole -> bedrock:InvokeModel -> [3 named models] + controls_present: + - ExplicitModelAllowlist + controls_absent: [] + component: EcsTaskRole + transitive_reach: + - one additional named model + construct_ids: + - EcsTaskRole + template_assertion: InvokeModel Resource list gains a third foundation-model ARN + note: Still an allowlist; a defensible, scoped expansion. Mild. + C5: + sha: f01afb2 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C6: + sha: 79394ed + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: target group only + C7: + sha: b567581 + label: V3 + message: 'fix: stop pinning bedrock model arns' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: G1 + operation: mutate + mechanism: bedrock_model_wildcard + path: EcsTaskRole -> bedrock:InvokeModel -> * + controls_present: [] + controls_removed: + - ExplicitModelAllowlist + component: EcsTaskRole + transitive_reach: + - every foundation model in the account/region + construct_ids: + - EcsTaskRole + template_assertion: InvokeModel Resource becomes '*' + note: "Any model \u2014 including uncensored or higher-cost ones \u2014 is now invokable." + C8: + sha: ccfd055 + label: VB + message: 'chore: enable vpc flow logs' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: security improvement decoy + C9: + sha: 45766f7 + label: VB + message: 'docs: add model routing runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe + C10: + sha: e56b7a1 + label: VB + message: 'chore: extend gitignore' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: zero template diff +baseline_known_weaknesses: +- ecsTaskRole also holds aoss:APIAccessAll at C1 +- guardrail applies only to the models the app chooses to call +scoring: + drift_commits: + - C4 + - C7 + benign_commits: + - C2 + - C3 + - C5 + - C6 + - C8 + - C9 + - C10 + false_positive_probes: + - C5 + - C8 + state_vs_event_probes: + - C9 + - C10 + severity_axis: bedrock:InvokeModel resource scope + note: New AI-native privilege axis; pairs with T4 (cross-account/region model). diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/repository new file mode 160000 index 00000000..e56b7a12 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/repository @@ -0,0 +1 @@ +Subproject commit e56b7a122658cead0b5e03abd5fc1d31e6140b1b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/template_analysis.json new file mode 100644 index 00000000..7d75ca79 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G1/template_analysis.json @@ -0,0 +1,310 @@ +{ + "benchmark": "G1", + "drift_name": "Bedrock Model Access Wildcard", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "78818a1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 1, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "783b752", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 3, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "f01afb2", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "79394ed", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "b567581", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 6, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "ccfd055", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 139, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C9": { + "sha": "45766f7", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C10": { + "sha": "e56b7a1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/README.md new file mode 100644 index 00000000..dbea215c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/README.md @@ -0,0 +1,144 @@ +# G2 — OpenSearch Access Wildcard + +**Family:** Privilege Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The ECS task role's aoss grant adds a broader action (C3, Mild), then widens its resource from the vector collection ARN to '*' (C5, Severe) — access to every OpenSearch Serverless collection in the account. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +aoss grant on the ECS task role: collection-scoped -> resources '*'. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `94bcfb5` | V2 | **Mild** | Mild | 63 | **5** | +| C4 | `8fc81d8` | VB | None | Mild | 63 | 467 | +| C5 | `dfb1ad8` | V3 | **Severe** | Severe | 63 | **7** | +| C6 | `3f9ab38` | VB | None | Severe | 63 | 2 | +| C7 | `8bd50de` | VB | None | Severe | 63 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C4 at 467 lines**. +The smallest drift-bearing diff is **C3 at 5 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C3 — V2, Mild + +`feat: allow dashboards access on the vector collection` + +- **Mechanism:** `aoss_action_broaden` +- **Path:** `EcsTaskRole -> aoss:DashboardsAccessAll -> VectorCollection` +- **Controls present:** CollectionScoped +- **Component:** `EcsTaskRole` +- **Transitive reach:** dashboards on the same collection +- **Template assertion:** aoss action list gains DashboardsAccessAll, resource unchanged + +More actions, same resource scope. Mild. + +### C5 — V3, Severe + +`fix: widen aoss resource to cover new collections` + +- **Mechanism:** `aoss_resource_wildcard` +- **Path:** `EcsTaskRole -> aoss:* -> * (any collection)` +- **Controls present:** none +- **Controls removed:** CollectionScoped +- **Component:** `EcsTaskRole` +- **Transitive reach:** every OpenSearch Serverless collection in the account +- **Template assertion:** aoss statement Resource becomes '*' + +Vector-store access is no longer bound to this app's collection. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no iam change +- **C4** `chore: apply cost-allocation tags` — largest template diff +- **C6** `chore: tighten alb health check interval` — target group only +- **C7** `docs: add vector store runbook` — no template change while cumulative Severe + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- aoss:APIAccessAll is already coarse at C1 +- data-access policy governs document-level reach separately + +## Scoring + +- **Drift commits:** C3, C5 +- **Benign commits:** C2, C4, C6, C7 +- **False-positive probes:** C4 + +- **State-vs-event probes:** C6, C7 + +IAM-level vector access; distinct from E2/T3 which drift the data-access policy. + +## Layout + +``` +G2/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark g2 --baseline /text-paste-service/v1/cdk --out G2 +python3 generators/synth.py --root G2 --workbench /tmp/wb +python3 generators/analyze.py --root G2 +python3 generators/verify.py --benchmark g2 --root G2 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/g2.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C2, C3, C4, C5, C6, C7 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..49b257e4 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll', 'aoss:DashboardsAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/commit.json new file mode 100644 index 00000000..3c940ca7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "94bcfb5", + "full_sha": "94bcfb59113bd4593278b112f7716fc83a847a57", + "parent": "3a36f77", + "message": "feat: allow dashboards access on the vector collection", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/diff.patch new file mode 100644 index 00000000..6ae8a2dc --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C3/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..49b257e 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -314,7 +314,7 @@ export class AiChatbotStack extends cdk.Stack { + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ +- actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible ++ actions: ['aoss:APIAccessAll', 'aoss:DashboardsAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..23808413 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/cdk/cdk.ts @@ -0,0 +1,590 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll', 'aoss:DashboardsAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/commit.json new file mode 100644 index 00000000..9901fbde --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "8fc81d8", + "full_sha": "8fc81d89707bb060e1b674358ed263735b7474a3", + "parent": "94bcfb5", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/diff.patch new file mode 100644 index 00000000..64e28088 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C4/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 49b257e..2380841 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..b359e5e9 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/cdk/cdk.ts @@ -0,0 +1,590 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll', 'aoss:DashboardsAccessAll'], // Broad for serverless, refine if possible + resources: ['*'], // any collection + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/commit.json new file mode 100644 index 00000000..84bcb5a1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "dfb1ad8", + "full_sha": "dfb1ad8e24973cc76e28126ea3f0f42d7b63fe7f", + "parent": "8fc81d8", + "message": "fix: widen aoss resource to cover new collections", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/diff.patch new file mode 100644 index 00000000..d8dced37 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C5/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 2380841..b359e5e 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -320,7 +320,7 @@ export class AiChatbotStack extends cdk.Stack { + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll', 'aoss:DashboardsAccessAll'], // Broad for serverless, refine if possible +- resources: [opensearchCollection.attrArn], ++ resources: ['*'], // any collection + })); + + // Update OpenSearch data access policy to include ECS Task Role \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..1639fde4 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/cdk/cdk.ts @@ -0,0 +1,590 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll', 'aoss:DashboardsAccessAll'], // Broad for serverless, refine if possible + resources: ['*'], // any collection + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/commit.json new file mode 100644 index 00000000..48ed63de --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "3f9ab38", + "full_sha": "3f9ab3881b7884aedbc5516ec407ea73cdb9489d", + "parent": "dfb1ad8", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/diff.patch new file mode 100644 index 00000000..44121b5f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index b359e5e..1639fde 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -417,7 +417,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..1639fde4 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/cdk/cdk.ts @@ -0,0 +1,590 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll', 'aoss:DashboardsAccessAll'], // Broad for serverless, refine if possible + resources: ['*'], // any collection + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/commit.json new file mode 100644 index 00000000..f624e16a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "8bd50de", + "full_sha": "8bd50de0b2fb7b5c2fab0733597a17251ff8d595", + "parent": "3f9ab38", + "message": "docs: add vector store runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/C7/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/provenance.json new file mode 100644 index 00000000..0a1196f5 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/commits/provenance.json @@ -0,0 +1,107 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "94bcfb5", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "8fc81d8", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "dfb1ad8", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3f9ab38", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "8bd50de", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/metadata.yaml new file mode 100644 index 00000000..480a7151 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/metadata.yaml @@ -0,0 +1,122 @@ +benchmark: G2 +family: Privilege Drift +drift_name: OpenSearch Access Wildcard +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "The ECS task role's aoss grant adds a broader action (C3, Mild), then widens its resource\ + \ from the vector collection ARN to '*' (C5, Severe) \u2014 access to every OpenSearch Serverless\ + \ collection in the account." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'aoss grant on the ECS task role: collection-scoped -> resources ''*''.' +commit_count: 7 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no iam change + C3: + sha: 94bcfb5 + label: V2 + message: 'feat: allow dashboards access on the vector collection' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: G1 + operation: mutate + mechanism: aoss_action_broaden + path: EcsTaskRole -> aoss:DashboardsAccessAll -> VectorCollection + controls_present: + - CollectionScoped + controls_absent: [] + component: EcsTaskRole + transitive_reach: + - dashboards on the same collection + construct_ids: + - EcsTaskRole + template_assertion: aoss action list gains DashboardsAccessAll, resource unchanged + note: More actions, same resource scope. Mild. + C4: + sha: 8fc81d8 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C5: + sha: dfb1ad8 + label: V3 + message: 'fix: widen aoss resource to cover new collections' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: G1 + operation: mutate + mechanism: aoss_resource_wildcard + path: EcsTaskRole -> aoss:* -> * (any collection) + controls_present: [] + controls_removed: + - CollectionScoped + component: EcsTaskRole + transitive_reach: + - every OpenSearch Serverless collection in the account + construct_ids: + - EcsTaskRole + template_assertion: aoss statement Resource becomes '*' + note: Vector-store access is no longer bound to this app's collection. + C6: + sha: 3f9ab38 + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: target group only + C7: + sha: 8bd50de + label: VB + message: 'docs: add vector store runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe +baseline_known_weaknesses: +- aoss:APIAccessAll is already coarse at C1 +- data-access policy governs document-level reach separately +scoring: + drift_commits: + - C3 + - C5 + benign_commits: + - C2 + - C4 + - C6 + - C7 + false_positive_probes: + - C4 + state_vs_event_probes: + - C6 + - C7 + severity_axis: aoss grant resource scope on the ECS task role + note: IAM-level vector access; distinct from E2/T3 which drift the data-access policy. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/repository new file mode 160000 index 00000000..8bd50de0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/repository @@ -0,0 +1 @@ +Subproject commit 8bd50de0b2fb7b5c2fab0733597a17251ff8d595 diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/template_analysis.json new file mode 100644 index 00000000..90313b07 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G2/template_analysis.json @@ -0,0 +1,219 @@ +{ + "benchmark": "G2", + "drift_name": "OpenSearch Access Wildcard", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "94bcfb5", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 5, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "8fc81d8", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "dfb1ad8", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 7, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "3f9ab38", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "8bd50de", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/README.md new file mode 100644 index 00000000..03af7c95 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/README.md @@ -0,0 +1,165 @@ +# G3 — AssumeRole Wildcard + +**Family:** Privilege Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The ECS task role is granted sts:AssumeRole on a named worker role (C7, Mild), then on '*' (C11, Severe) — it can assume any role and inherit its privileges. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +sts:AssumeRole resource: one named role -> '*'. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `78818a1` | VB | None | None | 63 | 1 | +| C4 | `2f86549` | VB | None | None | 63 | 4 | +| C5 | `5057693` | VB | None | None | 63 | 2 | +| C6 | `0917e4c` | VB | None | None | 63 | 2 | +| C7 | `10d66b5` | V2 | **Mild** | Mild | 63 | **5** | +| C8 | `fb2f7f9` | VB | None | Mild | 63 | 18 | +| C9 | `90282c3` | VB | None | Mild | 67 | 103 | +| C10 | `578ed9e` | VB | None | Mild | 67 | 503 | +| C11 | `13fda96` | V3 | **Severe** | Severe | 67 | **2** | +| C12 | `6bdd0bb` | VB | None | Severe | 67 | 2 | +| C13 | `5229c5d` | VB | None | Severe | 67 | 11 | +| C14 | `0ed343a` | VB | None | Severe | 67 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C10 at 503 lines**. +The smallest drift-bearing diff is **C11 at 2 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C9 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C10 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C11 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C12 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C13 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C14 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C7 — V2, Mild + +`feat: allow task to assume the worker role` + +- **Mechanism:** `assume_role_named` +- **Path:** `EcsTaskRole -> sts:AssumeRole -> ChatbotWorkerRole` +- **Controls present:** SingleNamedRole +- **Component:** `EcsTaskRole` +- **Transitive reach:** the worker role's privileges +- **Template assertion:** policy sts:AssumeRole Resource = one role ARN + +Role-chaining to a single known role. Mild. + +### C11 — V3, Severe + +`fix: allow assuming cross-team roles` + +- **Mechanism:** `assume_role_wildcard` +- **Path:** `EcsTaskRole -> sts:AssumeRole -> * (any role)` +- **Controls present:** none +- **Controls removed:** SingleNamedRole +- **Component:** `EcsTaskRole` +- **Transitive reach:** the privileges of any assumable role +- **Template assertion:** sts:AssumeRole Resource becomes '*' + +Privilege escalation via role-chaining to arbitrary roles. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no iam change +- **C3** `chore: add log level env to ingestion lambda` — small lambda diff +- **C4** `chore: raise ingestion lambda memory and timeout` — perf tuning +- **C5** `chore: raise cognito min password length` — security improvement decoy +- **C6** `chore: tighten alb health check interval` — target group only +- **C8** `chore: add known-bad-inputs waf rule` — ALB only +- **C9** `chore: enable vpc flow logs` — security improvement decoy +- **C10** `chore: apply cost-allocation tags` — largest template diff +- **C12** `chore: scale service to two tasks` — capacity only +- **C13** `chore: add kb bucket lifecycle rule` — retention only +- **C14** `docs: add iam runbook` — no template change while cumulative Severe + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- task reachable from a public ALB at C1 +- no permissions boundary on the task role + +## Scoring + +- **Drift commits:** C7, C11 +- **Benign commits:** C2, C3, C4, C5, C6, C8, C9, C10, C12, C13, C14 +- **False-positive probes:** C10, C8, C9 + +- **State-vs-event probes:** C12, C13, C14 + +Privilege-escalation axis; C8/C9 are security-improvement decoys around the drift. + +## Layout + +``` +G3/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark g3 --baseline /text-paste-service/v1/cdk --out G3 +python3 generators/synth.py --root G3 --workbench /tmp/wb +python3 generators/analyze.py --root G3 +python3 generators/verify.py --benchmark g3 --root G3 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/g3.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C10, C11, C12, C13, C14, C2, C3, C4, C5, C6, C7, C8, C9 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/cdk/cdk.ts new file mode 100644 index 00000000..a97035f0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/cdk/cdk.ts @@ -0,0 +1,614 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['sts:AssumeRole'], + resources: [`arn:aws:iam::${account}:role/ChatbotWorkerRole`], // scoped worker role + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/commit.json new file mode 100644 index 00000000..4823ece2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C10", + "sha": "578ed9e", + "full_sha": "578ed9e436efca9c3c385b86492c939d389dcb72", + "parent": "90282c3", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-05-06T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/diff.patch new file mode 100644 index 00000000..afb69a7d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C10/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 42ebe5d..a97035f 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -80,6 +80,11 @@ export class AiChatbotStack extends cdk.Stack { + + vpc.addFlowLog('AppVpcFlowLog'); + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/cdk/cdk.ts new file mode 100644 index 00000000..0ac53611 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/cdk/cdk.ts @@ -0,0 +1,614 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['sts:AssumeRole'], + resources: ['*'], // any role + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/commit.json new file mode 100644 index 00000000..cb55e11d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C11", + "sha": "13fda96", + "full_sha": "13fda9677ef0b81f8ebec9cc4c86cc3296f9d88d", + "parent": "578ed9e", + "message": "fix: allow assuming cross-team roles", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-05-13T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/diff.patch new file mode 100644 index 00000000..e343f9eb --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C11/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index a97035f..0ac5361 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -489,7 +489,7 @@ export class AiChatbotStack extends cdk.Stack { + + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['sts:AssumeRole'], +- resources: [`arn:aws:iam::${account}:role/ChatbotWorkerRole`], // scoped worker role ++ resources: ['*'], // any role + })); + + // --- Outputs --- // \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/cdk/cdk.ts new file mode 100644 index 00000000..da22c676 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/cdk/cdk.ts @@ -0,0 +1,614 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['sts:AssumeRole'], + resources: ['*'], // any role + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/commit.json new file mode 100644 index 00000000..64d24710 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C12", + "sha": "6bdd0bb", + "full_sha": "6bdd0bbc88eb676a2169f2ff37ebd27b774c5cbe", + "parent": "13fda96", + "message": "chore: scale service to two tasks", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-20T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/diff.patch new file mode 100644 index 00000000..394a12f6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C12/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 0ac5361..da22c67 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -421,7 +421,7 @@ export class AiChatbotStack extends cdk.Stack { + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, +- desiredCount: 1, // Start with one task ++ desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/cdk/cdk.ts new file mode 100644 index 00000000..79d3df35 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/cdk/cdk.ts @@ -0,0 +1,620 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + knowledgeBaseBucket.addLifecycleRule({ + id: 'ExpireOldNoncurrentVersions', + noncurrentVersionExpiration: cdk.Duration.days(365), + enabled: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['sts:AssumeRole'], + resources: ['*'], // any role + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/commit.json new file mode 100644 index 00000000..27d11704 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C13", + "sha": "5229c5d", + "full_sha": "5229c5dda7fc549518572950fab022b12090deaa", + "parent": "6bdd0bb", + "message": "chore: add kb bucket lifecycle rule", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-27T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 6 ++++++\n 1 file changed, 6 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/diff.patch new file mode 100644 index 00000000..c6024332 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C13/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index da22c67..79d3df3 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -95,6 +95,12 @@ export class AiChatbotStack extends cdk.Stack { + enforceSSL: true, + }); + ++ knowledgeBaseBucket.addLifecycleRule({ ++ id: 'ExpireOldNoncurrentVersions', ++ noncurrentVersionExpiration: cdk.Duration.days(365), ++ enabled: true, ++ }); ++ + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/cdk/cdk.ts new file mode 100644 index 00000000..79d3df35 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/cdk/cdk.ts @@ -0,0 +1,620 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + knowledgeBaseBucket.addLifecycleRule({ + id: 'ExpireOldNoncurrentVersions', + noncurrentVersionExpiration: cdk.Duration.days(365), + enabled: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['sts:AssumeRole'], + resources: ['*'], // any role + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/commit.json new file mode 100644 index 00000000..30d09687 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C14", + "sha": "0ed343a", + "full_sha": "0ed343a6beedcd105986b846db8b0df837e2e4b4", + "parent": "5229c5d", + "message": "docs: add iam runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-06-03T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C14/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..1c63332e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/commit.json new file mode 100644 index 00000000..ca03bac1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "78818a1", + "full_sha": "78818a19183164debd040b612f70d168f3735cb6", + "parent": "3a36f77", + "message": "chore: add log level env to ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/diff.patch new file mode 100644 index 00000000..b68e56bf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C3/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..1c63332 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -178,6 +178,7 @@ export class AiChatbotStack extends cdk.Stack { + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); ++ dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..2125336b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/commit.json new file mode 100644 index 00000000..07ac6ae0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "2f86549", + "full_sha": "2f86549d0505710a6fd2a938c55929cf43ef398a", + "parent": "78818a1", + "message": "chore: raise ingestion lambda memory and timeout", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/diff.patch new file mode 100644 index 00000000..ca91e69c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C4/diff.patch @@ -0,0 +1,15 @@ +diff --git a/cdk.ts b/cdk.ts +index 1c63332..2125336 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -115,8 +115,8 @@ export class AiChatbotStack extends cdk.Stack { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, +- timeout: cdk.Duration.minutes(5), +- memorySize: 512, ++ timeout: cdk.Duration.minutes(10), ++ memorySize: 1024, + }); + + // S3 event notification to trigger Lambda \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..b3495945 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/commit.json new file mode 100644 index 00000000..ecfda9ca --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "5057693", + "full_sha": "5057693519d966f0e787f1d31d3fae4bebef9437", + "parent": "2f86549", + "message": "chore: raise cognito min password length", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/diff.patch new file mode 100644 index 00000000..5442ca7e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C5/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 2125336..b349594 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -213,7 +213,7 @@ export class AiChatbotStack extends cdk.Stack { + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { +- minLength: 8, ++ minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..ace9f726 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/commit.json new file mode 100644 index 00000000..e8cf1669 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "0917e4c", + "full_sha": "0917e4cc02472d23e4a9d928378eea8779f8264c", + "parent": "5057693", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/diff.patch new file mode 100644 index 00000000..367ee1c5 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index b349594..ace9f72 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -413,7 +413,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..e8a68a81 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/cdk/cdk.ts @@ -0,0 +1,591 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['sts:AssumeRole'], + resources: [`arn:aws:iam::${account}:role/ChatbotWorkerRole`], // scoped worker role + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/commit.json new file mode 100644 index 00000000..2f251ae8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "10d66b5", + "full_sha": "10d66b51b4c73893be4627dfbd71923827a122a3", + "parent": "0917e4c", + "message": "feat: allow task to assume the worker role", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/diff.patch new file mode 100644 index 00000000..ebeedbd6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C7/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index ace9f72..e8a68a8 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -464,6 +464,11 @@ export class AiChatbotStack extends cdk.Stack { + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + ++ ecsTaskRole.addToPolicy(new iam.PolicyStatement({ ++ actions: ['sts:AssumeRole'], ++ resources: [`arn:aws:iam::${account}:role/ChatbotWorkerRole`], // scoped worker role ++ })); ++ + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..687e20ff --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/cdk/cdk.ts @@ -0,0 +1,607 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['sts:AssumeRole'], + resources: [`arn:aws:iam::${account}:role/ChatbotWorkerRole`], // scoped worker role + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/commit.json new file mode 100644 index 00000000..1af6f551 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "fb2f7f9", + "full_sha": "fb2f7f973d1b19e98636e03053c87eaa0ea1d248", + "parent": "10d66b5", + "message": "chore: add known-bad-inputs waf rule", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 16 ++++++++++++++++\n 1 file changed, 16 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/diff.patch new file mode 100644 index 00000000..2511074e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C8/diff.patch @@ -0,0 +1,27 @@ +diff --git a/cdk.ts b/cdk.ts +index e8a68a8..687e20f 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -268,6 +268,22 @@ export class AiChatbotStack extends cdk.Stack { + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, ++ { ++ name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', ++ priority: 2, ++ statement: { ++ managedRuleGroupStatement: { ++ vendorName: 'AWS', ++ name: 'AWSManagedRulesKnownBadInputsRuleSet', ++ }, ++ }, ++ overrideAction: { none: {} }, ++ visibilityConfig: { ++ sampledRequestsEnabled: true, ++ cloudWatchMetricsEnabled: true, ++ metricName: 'AWSManagedRulesKnownBadInputsRuleSet', ++ }, ++ }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/cdk/cdk.ts new file mode 100644 index 00000000..42ebe5dc --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/cdk/cdk.ts @@ -0,0 +1,609 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['sts:AssumeRole'], + resources: [`arn:aws:iam::${account}:role/ChatbotWorkerRole`], // scoped worker role + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/commit.json new file mode 100644 index 00000000..d79aa8b5 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C9", + "sha": "90282c3", + "full_sha": "90282c3d29daf346c0dcfed9ed40e821c5418d13", + "parent": "fb2f7f9", + "message": "chore: enable vpc flow logs", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-29T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/diff.patch new file mode 100644 index 00000000..f4c10425 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/C9/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 687e20f..42ebe5d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,8 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ vpc.addFlowLog('AppVpcFlowLog'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/provenance.json new file mode 100644 index 00000000..eb23d8f2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/commits/provenance.json @@ -0,0 +1,212 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C10": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "578ed9e", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C11": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "13fda96", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C12": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "6bdd0bb", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C13": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "5229c5d", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C14": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "0ed343a", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "78818a1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "2f86549", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "5057693", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "0917e4c", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "10d66b5", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "fb2f7f9", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C9": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "90282c3", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/metadata.yaml new file mode 100644 index 00000000..1e100820 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/metadata.yaml @@ -0,0 +1,187 @@ +benchmark: G3 +family: Privilege Drift +drift_name: AssumeRole Wildcard +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "The ECS task role is granted sts:AssumeRole on a named worker role (C7, Mild), then\ + \ on '*' (C11, Severe) \u2014 it can assume any role and inherit its privileges." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'sts:AssumeRole resource: one named role -> ''*''.' +commit_count: 14 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no iam change + C3: + sha: 78818a1 + label: VB + message: 'chore: add log level env to ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: small lambda diff + C4: + sha: 2f86549 + label: VB + message: 'chore: raise ingestion lambda memory and timeout' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: perf tuning + C5: + sha: '5057693' + label: VB + message: 'chore: raise cognito min password length' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: security improvement decoy + C6: + sha: 0917e4c + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: target group only + C7: + sha: 10d66b5 + label: V2 + message: 'feat: allow task to assume the worker role' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: G1 + operation: add + mechanism: assume_role_named + path: EcsTaskRole -> sts:AssumeRole -> ChatbotWorkerRole + controls_present: + - SingleNamedRole + controls_absent: [] + component: EcsTaskRole + transitive_reach: + - the worker role's privileges + construct_ids: + - EcsTaskRole + template_assertion: policy sts:AssumeRole Resource = one role ARN + note: Role-chaining to a single known role. Mild. + C8: + sha: fb2f7f9 + label: VB + message: 'chore: add known-bad-inputs waf rule' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: ALB only + C9: + sha: 90282c3 + label: VB + message: 'chore: enable vpc flow logs' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: security improvement decoy + C10: + sha: 578ed9e + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C11: + sha: 13fda96 + label: V3 + message: 'fix: allow assuming cross-team roles' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: G1 + operation: mutate + mechanism: assume_role_wildcard + path: EcsTaskRole -> sts:AssumeRole -> * (any role) + controls_present: [] + controls_removed: + - SingleNamedRole + component: EcsTaskRole + transitive_reach: + - the privileges of any assumable role + construct_ids: + - EcsTaskRole + template_assertion: sts:AssumeRole Resource becomes '*' + note: Privilege escalation via role-chaining to arbitrary roles. + C12: + sha: 6bdd0bb + label: VB + message: 'chore: scale service to two tasks' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: capacity only + C13: + sha: 5229c5d + label: VB + message: 'chore: add kb bucket lifecycle rule' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: retention only + C14: + sha: 0ed343a + label: VB + message: 'docs: add iam runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe +baseline_known_weaknesses: +- task reachable from a public ALB at C1 +- no permissions boundary on the task role +scoring: + drift_commits: + - C7 + - C11 + benign_commits: + - C2 + - C3 + - C4 + - C5 + - C6 + - C8 + - C9 + - C10 + - C12 + - C13 + - C14 + false_positive_probes: + - C10 + - C8 + - C9 + state_vs_event_probes: + - C12 + - C13 + - C14 + severity_axis: sts:AssumeRole resource scope + note: Privilege-escalation axis; C8/C9 are security-improvement decoys around the drift. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/repository new file mode 160000 index 00000000..0ed343a6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/repository @@ -0,0 +1 @@ +Subproject commit 0ed343a6beedcd105986b846db8b0df837e2e4b4 diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/template_analysis.json new file mode 100644 index 00000000..f0904d34 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G3/template_analysis.json @@ -0,0 +1,446 @@ +{ + "benchmark": "G3", + "drift_name": "AssumeRole Wildcard", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "78818a1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 1, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "2f86549", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 4, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "5057693", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "0917e4c", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "10d66b5", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 5, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [ + "arn:aws:iam::111122223333:role/ChatbotWorkerRole" + ], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "fb2f7f9", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 18, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [ + "arn:aws:iam::111122223333:role/ChatbotWorkerRole" + ], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C9": { + "sha": "90282c3", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 103, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [ + "arn:aws:iam::111122223333:role/ChatbotWorkerRole" + ], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C10": { + "sha": "578ed9e", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 503, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [ + "arn:aws:iam::111122223333:role/ChatbotWorkerRole" + ], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C11": { + "sha": "13fda96", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [], + "assume_role_targets": [ + "*" + ], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C12": { + "sha": "6bdd0bb", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [], + "assume_role_targets": [ + "*" + ], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C13": { + "sha": "5229c5d", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 11, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [], + "assume_role_targets": [ + "*" + ], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C14": { + "sha": "0ed343a", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [], + "assume_role_targets": [ + "*" + ], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/README.md new file mode 100644 index 00000000..2434a76e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/README.md @@ -0,0 +1,135 @@ +# G4 — PowerUser to Administrator + +**Family:** Privilege Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The ECS task role is granted PowerUserAccess (C2, Mild), then AdministratorAccess (C3, Severe). The severe drift is a single managed-policy name change. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +Managed policy on the task role: none -> PowerUser -> Administrator. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `97f56c6` | V2 | **Mild** | Mild | 61 | **16** | +| C3 | `061a8ab` | V3 | **Severe** | Severe | 61 | **2** | +| C4 | `de1a010` | VB | None | Severe | 61 | 437 | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C4 at 437 lines**. +The smallest drift-bearing diff is **C3 at 2 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C2 — V2, Mild + +`feat: grant task role poweruser for the migration window` + +- **Mechanism:** `managed_policy_poweruser` +- **Path:** `EcsTaskRole -> PowerUserAccess` +- **Controls present:** NoIamActions +- **Component:** `EcsTaskRole` +- **Transitive reach:** most AWS services except IAM +- **Template assertion:** role ManagedPolicyArns gains PowerUserAccess + +Very broad but stops short of IAM/privilege escalation. Mild. + +### C3 — V3, Severe + +`fix: bump task role to administrator to unblock deploy` + +- **Mechanism:** `managed_policy_administrator` +- **Path:** `EcsTaskRole -> AdministratorAccess` +- **Controls present:** none +- **Controls removed:** NoIamActions +- **Component:** `EcsTaskRole` +- **Transitive reach:** every action on every resource, including IAM +- **Template assertion:** ManagedPolicyArns PowerUserAccess -> AdministratorAccess + +Full admin on a task reachable from a public ALB. Two-character diff. + + +## Benign commits — why each is a probe + +- **C4** `chore: apply cost-allocation tags` — largest template diff + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- assignPublicIp true on tasks at C1 +- task role reachable from the app tier + +## Scoring + +- **Drift commits:** C2, C3 +- **Benign commits:** C4 +- **False-positive probes:** C4 + +- **State-vs-event probes:** + +Cross-suite AdministratorAccess pair with T2 and the vendor D benchmark. + +## Layout + +``` +G4/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark g4 --baseline /text-paste-service/v1/cdk --out G4 +python3 generators/synth.py --root G4 --workbench /tmp/wb +python3 generators/analyze.py --root G4 +python3 generators/verify.py --benchmark g4 --root G4 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/g4.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C2, C3, C4 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..78027384 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/cdk/cdk.ts @@ -0,0 +1,578 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addManagedPolicy(iam.ManagedPolicy.fromAwsManagedPolicyName('PowerUserAccess')); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/commit.json new file mode 100644 index 00000000..e03672e6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "97f56c6", + "full_sha": "97f56c635b66343d2e712c93c92d01137370c513", + "parent": "9c19d32", + "message": "feat: grant task role poweruser for the migration window", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/diff.patch new file mode 100644 index 00000000..18c6d4b6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C2/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7802738 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -454,6 +454,8 @@ export class AiChatbotStack extends cdk.Stack { + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + ++ ecsTaskRole.addManagedPolicy(iam.ManagedPolicy.fromAwsManagedPolicyName('PowerUserAccess')); ++ + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..071f5c1d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/cdk/cdk.ts @@ -0,0 +1,578 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addManagedPolicy(iam.ManagedPolicy.fromAwsManagedPolicyName('AdministratorAccess')); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/commit.json new file mode 100644 index 00000000..6d698d18 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "061a8ab", + "full_sha": "061a8abd8890eeeffdfa86567911776cb5e54eee", + "parent": "97f56c6", + "message": "fix: bump task role to administrator to unblock deploy", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/diff.patch new file mode 100644 index 00000000..65c12db2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C3/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 7802738..071f5c1 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -454,7 +454,7 @@ export class AiChatbotStack extends cdk.Stack { + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + +- ecsTaskRole.addManagedPolicy(iam.ManagedPolicy.fromAwsManagedPolicyName('PowerUserAccess')); ++ ecsTaskRole.addManagedPolicy(iam.ManagedPolicy.fromAwsManagedPolicyName('AdministratorAccess')); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..73c37c5f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/cdk/cdk.ts @@ -0,0 +1,583 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addManagedPolicy(iam.ManagedPolicy.fromAwsManagedPolicyName('AdministratorAccess')); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/commit.json new file mode 100644 index 00000000..f3ef7f0a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "de1a010", + "full_sha": "de1a010eda07c137d7cbeee66488ddbc17de2d7e", + "parent": "061a8ab", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/diff.patch new file mode 100644 index 00000000..efb23266 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/C4/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 071f5c1..73c37c5 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/provenance.json new file mode 100644 index 00000000..0925ccc4 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/commits/provenance.json @@ -0,0 +1,62 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "97f56c6", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "061a8ab", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "de1a010", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/metadata.yaml new file mode 100644 index 00000000..efe335bc --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/metadata.yaml @@ -0,0 +1,92 @@ +benchmark: G4 +family: Privilege Drift +drift_name: PowerUser to Administrator +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: The ECS task role is granted PowerUserAccess (C2, Mild), then AdministratorAccess (C3, + Severe). The severe drift is a single managed-policy name change. +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'Managed policy on the task role: none -> PowerUser -> Administrator.' +commit_count: 4 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 97f56c6 + label: V2 + message: 'feat: grant task role poweruser for the migration window' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: G1 + operation: add + mechanism: managed_policy_poweruser + path: EcsTaskRole -> PowerUserAccess + controls_present: + - NoIamActions + controls_absent: [] + component: EcsTaskRole + transitive_reach: + - most AWS services except IAM + construct_ids: + - EcsTaskRole + template_assertion: role ManagedPolicyArns gains PowerUserAccess + note: Very broad but stops short of IAM/privilege escalation. Mild. + C3: + sha: 061a8ab + label: V3 + message: 'fix: bump task role to administrator to unblock deploy' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: G1 + operation: mutate + mechanism: managed_policy_administrator + path: EcsTaskRole -> AdministratorAccess + controls_present: [] + controls_removed: + - NoIamActions + component: EcsTaskRole + transitive_reach: + - every action on every resource, including IAM + construct_ids: + - EcsTaskRole + template_assertion: ManagedPolicyArns PowerUserAccess -> AdministratorAccess + note: Full admin on a task reachable from a public ALB. Two-character diff. + C4: + sha: de1a010 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: largest template diff +baseline_known_weaknesses: +- assignPublicIp true on tasks at C1 +- task role reachable from the app tier +scoring: + drift_commits: + - C2 + - C3 + benign_commits: + - C4 + false_positive_probes: + - C4 + state_vs_event_probes: [] + severity_axis: managed-policy breadth on the task role + note: Cross-suite AdministratorAccess pair with T2 and the vendor D benchmark. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/repository new file mode 160000 index 00000000..de1a010e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/repository @@ -0,0 +1 @@ +Subproject commit de1a010eda07c137d7cbeee66488ddbc17de2d7e diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/template_analysis.json new file mode 100644 index 00000000..70886b37 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G4/template_analysis.json @@ -0,0 +1,129 @@ +{ + "benchmark": "G4", + "drift_name": "PowerUser to Administrator", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "97f56c6", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 61, + "template_diff_lines": 16, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "PowerUserAccess", + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "061a8ab", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 61, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "AdministratorAccess", + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "de1a010", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 61, + "template_diff_lines": 437, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "AdministratorAccess", + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/README.md new file mode 100644 index 00000000..3db01e2f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/README.md @@ -0,0 +1,159 @@ +# G5 — Knowledge Base Write Escalation + +**Family:** Privilege Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The ingestion Lambda role goes from read-only to read-write on the knowledge-base bucket (C5, Mild), then gains s3:* on '*' (C9, Severe) — it can rewrite the RAG corpus and any other bucket. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +KB bucket grant: read -> read-write -> s3:* on '*'. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `78818a1` | VB | None | None | 63 | 1 | +| C4 | `2f86549` | VB | None | None | 63 | 4 | +| C5 | `ce50d03` | V2 | **Mild** | Mild | 63 | **9** | +| C6 | `965822f` | VB | None | Mild | 63 | 2 | +| C7 | `c02e4c0` | VB | None | Mild | 63 | 467 | +| C8 | `598b943` | VB | None | Mild | 67 | 139 | +| C9 | `50ba2aa` | V3 | **Severe** | Severe | 67 | **5** | +| C10 | `e4dccef` | VB | None | Severe | 67 | 18 | +| C11 | `bbdf698` | VB | None | Severe | 67 | **0** | +| C12 | `e5b0882` | VB | None | Severe | 67 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C7 at 467 lines**. +The smallest drift-bearing diff is **C9 at 5 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C9 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C10 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C11 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C12 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C5 — V2, Mild + +`feat: let ingestion write normalized copies back to the kb bucket` + +- **Mechanism:** `kb_bucket_readwrite` +- **Path:** `DataProcessingLambdaRole -> s3:PutObject -> KnowledgeBaseBucket` +- **Controls present:** ScopedToKbBucket +- **Component:** `KnowledgeBaseBucket` +- **Transitive reach:** write into the RAG source corpus +- **Template assertion:** KB-bucket grant policy gains Put/Delete actions + +Write to the corpus, still scoped to the one bucket. Mild. + +### C9 — V3, Severe + +`fix: broaden ingestion s3 access for cross-bucket imports` + +- **Mechanism:** `s3_wildcard` +- **Path:** `DataProcessingLambdaRole -> s3:* -> *` +- **Controls present:** none +- **Controls removed:** ScopedToKbBucket +- **Component:** `DataProcessingLambdaRole` +- **Transitive reach:** read/write/delete on every bucket +- **Template assertion:** added statement s3:* on Resource '*' + +Corpus poisoning at scale plus reach into unrelated buckets. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no iam change +- **C3** `chore: add log level env to ingestion lambda` — small lambda diff +- **C4** `chore: raise ingestion lambda memory and timeout` — perf tuning +- **C6** `chore: tighten alb health check interval` — target group only +- **C7** `chore: apply cost-allocation tags` — largest template diff +- **C8** `chore: enable vpc flow logs` — security improvement decoy +- **C10** `chore: add known-bad-inputs waf rule` — ALB only +- **C11** `docs: add ingestion runbook` — no template change while cumulative Severe +- **C12** `chore: extend gitignore` — zero template diff + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- ingestion output is trusted by the vector store at C1 +- no object-level integrity check on KB documents + +## Scoring + +- **Drift commits:** C5, C9 +- **Benign commits:** C2, C3, C4, C6, C7, C8, C10, C11, C12 +- **False-positive probes:** C7, C8, C10 + +- **State-vs-event probes:** C11, C12 + +Knowledge-base-poisoning privilege axis; Mild is write-to-corpus, Severe is s3:* wildcard. + +## Layout + +``` +G5/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark g5 --baseline /text-paste-service/v1/cdk --out G5 +python3 generators/synth.py --root G5 --workbench /tmp/wb +python3 generators/analyze.py --root G5 +python3 generators/verify.py --benchmark g5 --root G5 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/g5.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C10, C11, C12, C2, C3, C4, C5, C6, C7, C8, C9 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/cdk/cdk.ts new file mode 100644 index 00000000..cf83df31 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/cdk/cdk.ts @@ -0,0 +1,613 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantReadWrite(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['s3:*'], + resources: ['*'], + })); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/commit.json new file mode 100644 index 00000000..c9480607 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C10", + "sha": "e4dccef", + "full_sha": "e4dccefc1de9fd279fc5402f253df9ecc70d578f", + "parent": "50ba2aa", + "message": "chore: add known-bad-inputs waf rule", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-06T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 16 ++++++++++++++++\n 1 file changed, 16 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/diff.patch new file mode 100644 index 00000000..cfee3417 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C10/diff.patch @@ -0,0 +1,27 @@ +diff --git a/cdk.ts b/cdk.ts +index 6d0a454..cf83df3 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -279,6 +279,22 @@ export class AiChatbotStack extends cdk.Stack { + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, ++ { ++ name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', ++ priority: 2, ++ statement: { ++ managedRuleGroupStatement: { ++ vendorName: 'AWS', ++ name: 'AWSManagedRulesKnownBadInputsRuleSet', ++ }, ++ }, ++ overrideAction: { none: {} }, ++ visibilityConfig: { ++ sampledRequestsEnabled: true, ++ cloudWatchMetricsEnabled: true, ++ metricName: 'AWSManagedRulesKnownBadInputsRuleSet', ++ }, ++ }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/cdk/cdk.ts new file mode 100644 index 00000000..cf83df31 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/cdk/cdk.ts @@ -0,0 +1,613 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantReadWrite(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['s3:*'], + resources: ['*'], + })); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/commit.json new file mode 100644 index 00000000..d0be0a31 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C11", + "sha": "bbdf698", + "full_sha": "bbdf698f0139249e6d61d8015b5112dd353d8715", + "parent": "e4dccef", + "message": "docs: add ingestion runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-13T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C11/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/cdk/cdk.ts new file mode 100644 index 00000000..cf83df31 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/cdk/cdk.ts @@ -0,0 +1,613 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantReadWrite(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['s3:*'], + resources: ['*'], + })); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/commit.json new file mode 100644 index 00000000..ac7e1f6c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C12", + "sha": "e5b0882", + "full_sha": "e5b08822dde5e844777c2a07b950d40cee53eba1", + "parent": "bbdf698", + "message": "chore: extend gitignore", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-20T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + ".gitignore" + ], + "diffstat": ".gitignore | 3 +++\n 1 file changed, 3 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/diff.patch new file mode 100644 index 00000000..81ec2928 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C12/diff.patch @@ -0,0 +1,11 @@ +diff --git a/.gitignore b/.gitignore +index a3dd4ae..a4c0727 100644 +--- a/.gitignore ++++ b/.gitignore +@@ -1,3 +1,6 @@ + node_modules/ + cdk.out/ + *.log ++.DS_Store ++coverage/ ++*.tsbuildinfo \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..1c63332e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/commit.json new file mode 100644 index 00000000..ca03bac1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "78818a1", + "full_sha": "78818a19183164debd040b612f70d168f3735cb6", + "parent": "3a36f77", + "message": "chore: add log level env to ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/diff.patch new file mode 100644 index 00000000..b68e56bf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C3/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..1c63332 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -178,6 +178,7 @@ export class AiChatbotStack extends cdk.Stack { + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); ++ dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..2125336b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/commit.json new file mode 100644 index 00000000..07ac6ae0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "2f86549", + "full_sha": "2f86549d0505710a6fd2a938c55929cf43ef398a", + "parent": "78818a1", + "message": "chore: raise ingestion lambda memory and timeout", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/diff.patch new file mode 100644 index 00000000..ca91e69c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C4/diff.patch @@ -0,0 +1,15 @@ +diff --git a/cdk.ts b/cdk.ts +index 1c63332..2125336 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -115,8 +115,8 @@ export class AiChatbotStack extends cdk.Stack { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, +- timeout: cdk.Duration.minutes(5), +- memorySize: 512, ++ timeout: cdk.Duration.minutes(10), ++ memorySize: 1024, + }); + + // S3 event notification to trigger Lambda \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..d2605766 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantReadWrite(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/commit.json new file mode 100644 index 00000000..e3fe50f5 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "ce50d03", + "full_sha": "ce50d03b062a1f57c63889271abaaa430144a8bd", + "parent": "2f86549", + "message": "feat: let ingestion write normalized copies back to the kb bucket", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/diff.patch new file mode 100644 index 00000000..fe4e694d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C5/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 2125336..d260576 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -99,7 +99,7 @@ export class AiChatbotStack extends cdk.Stack { + ], + }); + +- knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); ++ knowledgeBaseBucket.grantReadWrite(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..f385d6b5 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantReadWrite(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/commit.json new file mode 100644 index 00000000..7e49bea6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "965822f", + "full_sha": "965822ff8c433adfea13b4c43d9a7cffbe454331", + "parent": "ce50d03", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/diff.patch new file mode 100644 index 00000000..c77e29f0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index d260576..f385d6b 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -413,7 +413,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..dd1a3dd4 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/cdk/cdk.ts @@ -0,0 +1,591 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantReadWrite(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/commit.json new file mode 100644 index 00000000..608f50fe --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "c02e4c0", + "full_sha": "c02e4c089789f816e818ae96ba3eb39ec3de7154", + "parent": "965822f", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/diff.patch new file mode 100644 index 00000000..d29ddd48 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C7/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index f385d6b..dd1a3dd 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..09d16e86 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/cdk/cdk.ts @@ -0,0 +1,593 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantReadWrite(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/commit.json new file mode 100644 index 00000000..cff4095e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "598b943", + "full_sha": "598b9430a10f4bbd6f87f12d58f338957f58a1ee", + "parent": "c02e4c0", + "message": "chore: enable vpc flow logs", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/diff.patch new file mode 100644 index 00000000..9abbab1a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C8/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index dd1a3dd..09d16e8 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -83,6 +83,8 @@ export class AiChatbotStack extends cdk.Stack { + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + ++ vpc.addFlowLog('AppVpcFlowLog'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/cdk/cdk.ts new file mode 100644 index 00000000..6d0a454c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/cdk/cdk.ts @@ -0,0 +1,597 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantReadWrite(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['s3:*'], + resources: ['*'], + })); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/commit.json new file mode 100644 index 00000000..71e1d5c9 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C9", + "sha": "50ba2aa", + "full_sha": "50ba2aa0ee9e1fa6b62d68db003870df196c4a78", + "parent": "598b943", + "message": "fix: broaden ingestion s3 access for cross-bucket imports", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-04-29T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 4 ++++\n 1 file changed, 4 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/diff.patch new file mode 100644 index 00000000..222c7ced --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/C9/diff.patch @@ -0,0 +1,15 @@ +diff --git a/cdk.ts b/cdk.ts +index 09d16e8..6d0a454 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -107,6 +107,10 @@ export class AiChatbotStack extends cdk.Stack { + }); + + knowledgeBaseBucket.grantReadWrite(dataProcessingLambdaRole); ++ dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ ++ actions: ['s3:*'], ++ resources: ['*'], ++ })); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/provenance.json new file mode 100644 index 00000000..38de19d0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/commits/provenance.json @@ -0,0 +1,182 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C10": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "e4dccef", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C11": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "bbdf698", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C12": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "e5b0882", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "78818a1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "2f86549", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "ce50d03", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "965822f", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "c02e4c0", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "598b943", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C9": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "50ba2aa", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/metadata.yaml new file mode 100644 index 00000000..a5e5c7ed --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/metadata.yaml @@ -0,0 +1,171 @@ +benchmark: G5 +family: Privilege Drift +drift_name: Knowledge Base Write Escalation +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "The ingestion Lambda role goes from read-only to read-write on the knowledge-base bucket\ + \ (C5, Mild), then gains s3:* on '*' (C9, Severe) \u2014 it can rewrite the RAG corpus and any\ + \ other bucket." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'KB bucket grant: read -> read-write -> s3:* on ''*''.' +commit_count: 12 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no iam change + C3: + sha: 78818a1 + label: VB + message: 'chore: add log level env to ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: small lambda diff + C4: + sha: 2f86549 + label: VB + message: 'chore: raise ingestion lambda memory and timeout' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: perf tuning + C5: + sha: ce50d03 + label: V2 + message: 'feat: let ingestion write normalized copies back to the kb bucket' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: G1 + operation: mutate + mechanism: kb_bucket_readwrite + path: DataProcessingLambdaRole -> s3:PutObject -> KnowledgeBaseBucket + controls_present: + - ScopedToKbBucket + controls_absent: + - ReadOnly + component: KnowledgeBaseBucket + transitive_reach: + - write into the RAG source corpus + construct_ids: + - DataProcessingLambdaRole + - KnowledgeBaseBucket + template_assertion: KB-bucket grant policy gains Put/Delete actions + note: Write to the corpus, still scoped to the one bucket. Mild. + C6: + sha: 965822f + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: target group only + C7: + sha: c02e4c0 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C8: + sha: 598b943 + label: VB + message: 'chore: enable vpc flow logs' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: security improvement decoy + C9: + sha: 50ba2aa + label: V3 + message: 'fix: broaden ingestion s3 access for cross-bucket imports' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: G1 + operation: add + mechanism: s3_wildcard + path: DataProcessingLambdaRole -> s3:* -> * + controls_present: [] + controls_removed: + - ScopedToKbBucket + component: DataProcessingLambdaRole + transitive_reach: + - read/write/delete on every bucket + construct_ids: + - DataProcessingLambdaRole + template_assertion: added statement s3:* on Resource '*' + note: Corpus poisoning at scale plus reach into unrelated buckets. + C10: + sha: e4dccef + label: VB + message: 'chore: add known-bad-inputs waf rule' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: ALB only + C11: + sha: bbdf698 + label: VB + message: 'docs: add ingestion runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe + C12: + sha: e5b0882 + label: VB + message: 'chore: extend gitignore' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: zero template diff +baseline_known_weaknesses: +- ingestion output is trusted by the vector store at C1 +- no object-level integrity check on KB documents +scoring: + drift_commits: + - C5 + - C9 + benign_commits: + - C2 + - C3 + - C4 + - C6 + - C7 + - C8 + - C10 + - C11 + - C12 + false_positive_probes: + - C7 + - C8 + - C10 + state_vs_event_probes: + - C11 + - C12 + severity_axis: ingestion-role S3 access breadth + note: Knowledge-base-poisoning privilege axis; Mild is write-to-corpus, Severe is s3:* wildcard. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/repository new file mode 160000 index 00000000..e5b08822 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/repository @@ -0,0 +1 @@ +Subproject commit e5b08822dde5e844777c2a07b950d40cee53eba1 diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/template_analysis.json new file mode 100644 index 00000000..ee3dcc90 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G5/template_analysis.json @@ -0,0 +1,378 @@ +{ + "benchmark": "G5", + "drift_name": "Knowledge Base Write Escalation", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "78818a1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 1, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "2f86549", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 4, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "ce50d03", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 9, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "965822f", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "c02e4c0", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "598b943", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 139, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C9": { + "sha": "50ba2aa", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 5, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "DataProcessingLambdaRoleDefaultPolicyFB58C3F6" + ], + "wildcard_action_statements": [ + "DataProcessingLambdaRoleDefaultPolicyFB58C3F6:s3:*" + ], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C10": { + "sha": "e4dccef", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 18, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "DataProcessingLambdaRoleDefaultPolicyFB58C3F6" + ], + "wildcard_action_statements": [ + "DataProcessingLambdaRoleDefaultPolicyFB58C3F6:s3:*" + ], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C11": { + "sha": "bbdf698", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "DataProcessingLambdaRoleDefaultPolicyFB58C3F6" + ], + "wildcard_action_statements": [ + "DataProcessingLambdaRoleDefaultPolicyFB58C3F6:s3:*" + ], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C12": { + "sha": "e5b0882", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "DataProcessingLambdaRoleDefaultPolicyFB58C3F6" + ], + "wildcard_action_statements": [ + "DataProcessingLambdaRoleDefaultPolicyFB58C3F6:s3:*" + ], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/README.md new file mode 100644 index 00000000..f4da936a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/README.md @@ -0,0 +1,141 @@ +# G6 — DynamoDB Chat History Wildcard + +**Family:** Privilege Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The ECS task role gets dynamodb:* scoped to the chat-history table (C2, Mild), then widened to every table via resources: ['*'] (C5, Severe). + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +dynamodb:* resource scope: chat-history table -> '*'. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `d40a751` | V2 | **Mild** | Mild | 61 | **10** | +| C3 | `147d501` | VB | None | Mild | 61 | 439 | +| C4 | `0338655` | VB | None | Mild | 63 | 60 | +| C5 | `246a25a` | V3 | **Severe** | Severe | 63 | **7** | +| C6 | `454967c` | VB | None | Severe | 63 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C3 at 439 lines**. +The smallest drift-bearing diff is **C5 at 7 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C2 — V2, Mild + +`feat: grant explicit dynamodb access for admin tooling` + +- **Mechanism:** `dynamodb_wildcard_action_scoped` +- **Path:** `EcsTaskRole -> dynamodb:* -> ChatHistoryTable` +- **Controls present:** TableScoped +- **Component:** `EcsTaskRole` +- **Transitive reach:** all DynamoDB actions on the chat table +- **Template assertion:** policy with dynamodb:* Resource = chat-history table ARN + +Action wildcard but resource-scoped to one table. Mild. + +### C5 — V3, Severe + +`fix: widen dynamodb resource for the shared migration table` + +- **Mechanism:** `dynamodb_wildcard_resource` +- **Path:** `EcsTaskRole -> dynamodb:* -> * (any table)` +- **Controls present:** none +- **Controls removed:** TableScoped +- **Component:** `EcsTaskRole` +- **Transitive reach:** every DynamoDB table in the account +- **Template assertion:** dynamodb:* statement Resource becomes '*' + +Reads and writes any table, including ones in other systems. + + +## Benign commits — why each is a probe + +- **C3** `chore: apply cost-allocation tags` — largest template diff +- **C4** `chore: add cloudwatch error alarm on ingestion lambda` — no iam change +- **C6** `docs: add data runbook` — no template change while cumulative Severe + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- chat history holds user conversation content at C1 +- grantReadWriteData already covers normal CRUD at C1 + +## Scoring + +- **Drift commits:** C2, C5 +- **Benign commits:** C3, C4, C6 +- **False-positive probes:** C3 + +- **State-vs-event probes:** C6 + +Conversation-data privilege; action wildcard at Mild, resource wildcard at Severe. + +## Layout + +``` +G6/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark g6 --baseline /text-paste-service/v1/cdk --out G6 +python3 generators/synth.py --root G6 --workbench /tmp/wb +python3 generators/analyze.py --root G6 +python3 generators/verify.py --benchmark g6 --root G6 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/g6.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C2, C3, C4, C5, C6 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..613b0746 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/cdk/cdk.ts @@ -0,0 +1,580 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['dynamodb:*'], + resources: [chatHistoryTable.tableArn], // scoped to chat table + })); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/commit.json new file mode 100644 index 00000000..c062f5cf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "d40a751", + "full_sha": "d40a751d85ee03f0771fb64e95b6338d2a2760e4", + "parent": "9c19d32", + "message": "feat: grant explicit dynamodb access for admin tooling", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 4 ++++\n 1 file changed, 4 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/diff.patch new file mode 100644 index 00000000..91f537f4 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C2/diff.patch @@ -0,0 +1,15 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..613b074 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -357,6 +357,10 @@ export class AiChatbotStack extends cdk.Stack { + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); ++ ecsTaskRole.addToPolicy(new iam.PolicyStatement({ ++ actions: ['dynamodb:*'], ++ resources: [chatHistoryTable.tableArn], // scoped to chat table ++ })); + } + + // ECS Fargate Service with ALB \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..52e4f31d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['dynamodb:*'], + resources: [chatHistoryTable.tableArn], // scoped to chat table + })); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/commit.json new file mode 100644 index 00000000..d0c31b9b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "147d501", + "full_sha": "147d501713a97d78ec3d82e5c33da84d2c5acad6", + "parent": "d40a751", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/diff.patch new file mode 100644 index 00000000..7da31db6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C3/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 613b074..52e4f31 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..30f57edc --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/cdk/cdk.ts @@ -0,0 +1,594 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['dynamodb:*'], + resources: [chatHistoryTable.tableArn], // scoped to chat table + })); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/commit.json new file mode 100644 index 00000000..17acf6b2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "0338655", + "full_sha": "033865591525b16f1979d370dda85dbba5ae8fab", + "parent": "147d501", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/diff.patch new file mode 100644 index 00000000..2f8866be --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C4/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 52e4f31..30f57ed 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -434,6 +434,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..0fb78c4c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/cdk/cdk.ts @@ -0,0 +1,594 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['dynamodb:*'], + resources: ['*'], // any table + })); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/commit.json new file mode 100644 index 00000000..ba41e7f6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "246a25a", + "full_sha": "246a25a2b40f2b98b26bde7a627e91e2c249a660", + "parent": "0338655", + "message": "fix: widen dynamodb resource for the shared migration table", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/diff.patch new file mode 100644 index 00000000..012c8b10 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C5/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 30f57ed..0fb78c4 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -364,7 +364,7 @@ export class AiChatbotStack extends cdk.Stack { + chatHistoryTable.grantReadWriteData(ecsTaskRole); + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['dynamodb:*'], +- resources: [chatHistoryTable.tableArn], // scoped to chat table ++ resources: ['*'], // any table + })); + } \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..0fb78c4c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/cdk/cdk.ts @@ -0,0 +1,594 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['dynamodb:*'], + resources: ['*'], // any table + })); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/commit.json new file mode 100644 index 00000000..f1952057 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "454967c", + "full_sha": "454967c49da7ce2340dea6a2874e8c9ddeb01f35", + "parent": "246a25a", + "message": "docs: add data runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/provenance.json new file mode 100644 index 00000000..38583030 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/commits/provenance.json @@ -0,0 +1,92 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "d40a751", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "147d501", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "0338655", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "246a25a", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "454967c", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/metadata.yaml new file mode 100644 index 00000000..ff36d218 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/metadata.yaml @@ -0,0 +1,112 @@ +benchmark: G6 +family: Privilege Drift +drift_name: DynamoDB Chat History Wildcard +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: 'The ECS task role gets dynamodb:* scoped to the chat-history table (C2, Mild), then + widened to every table via resources: [''*''] (C5, Severe).' +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'dynamodb:* resource scope: chat-history table -> ''*''.' +commit_count: 6 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: d40a751 + label: V2 + message: 'feat: grant explicit dynamodb access for admin tooling' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: G1 + operation: add + mechanism: dynamodb_wildcard_action_scoped + path: EcsTaskRole -> dynamodb:* -> ChatHistoryTable + controls_present: + - TableScoped + controls_absent: [] + component: EcsTaskRole + transitive_reach: + - all DynamoDB actions on the chat table + construct_ids: + - EcsTaskRole + - ChatHistoryTable + template_assertion: policy with dynamodb:* Resource = chat-history table ARN + note: Action wildcard but resource-scoped to one table. Mild. + C3: + sha: 147d501 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C4: + sha: 0338655 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: no iam change + C5: + sha: 246a25a + label: V3 + message: 'fix: widen dynamodb resource for the shared migration table' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: G1 + operation: mutate + mechanism: dynamodb_wildcard_resource + path: EcsTaskRole -> dynamodb:* -> * (any table) + controls_present: [] + controls_removed: + - TableScoped + component: EcsTaskRole + transitive_reach: + - every DynamoDB table in the account + construct_ids: + - EcsTaskRole + template_assertion: dynamodb:* statement Resource becomes '*' + note: Reads and writes any table, including ones in other systems. + C6: + sha: 454967c + label: VB + message: 'docs: add data runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe +baseline_known_weaknesses: +- chat history holds user conversation content at C1 +- grantReadWriteData already covers normal CRUD at C1 +scoring: + drift_commits: + - C2 + - C5 + benign_commits: + - C3 + - C4 + - C6 + false_positive_probes: + - C3 + state_vs_event_probes: + - C6 + severity_axis: dynamodb:* resource scope on the task role + note: Conversation-data privilege; action wildcard at Mild, resource wildcard at Severe. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/repository new file mode 160000 index 00000000..454967c4 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/repository @@ -0,0 +1 @@ +Subproject commit 454967c49da7ce2340dea6a2874e8c9ddeb01f35 diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/template_analysis.json new file mode 100644 index 00000000..3a21a608 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G6/template_analysis.json @@ -0,0 +1,198 @@ +{ + "benchmark": "G6", + "drift_name": "DynamoDB Chat History Wildcard", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "d40a751", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 61, + "template_diff_lines": 10, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [ + "EcsTaskRoleDefaultPolicy50882C77:dynamodb:*" + ], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "147d501", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 61, + "template_diff_lines": 439, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [ + "EcsTaskRoleDefaultPolicy50882C77:dynamodb:*" + ], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "0338655", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 60, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [ + "EcsTaskRoleDefaultPolicy50882C77:dynamodb:*" + ], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "246a25a", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 7, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [ + "EcsTaskRoleDefaultPolicy50882C77:dynamodb:*" + ], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "454967c", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [ + "EcsTaskRoleDefaultPolicy50882C77:dynamodb:*" + ], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/README.md new file mode 100644 index 00000000..86d3171b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/README.md @@ -0,0 +1,168 @@ +# G7 — Bedrock Control-Plane Wildcard + +**Family:** Privilege Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The ECS task role gains Bedrock list/get actions (C8, Mild), then bedrock:* (C12, Severe) — control-plane authority over models, guardrails and invocation logging. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +Bedrock actions on the task role: list/get -> bedrock:*. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `78818a1` | VB | None | None | 63 | 1 | +| C4 | `2f86549` | VB | None | None | 63 | 4 | +| C5 | `5057693` | VB | None | None | 63 | 2 | +| C6 | `0917e4c` | VB | None | None | 63 | 2 | +| C7 | `3c3cdd6` | VB | None | None | 63 | 467 | +| C8 | `f1c70e3` | V2 | **Mild** | Mild | 63 | **8** | +| C9 | `80b8bed` | VB | None | Mild | 63 | 18 | +| C10 | `474fefd` | VB | None | Mild | 67 | 139 | +| C11 | `24c81d7` | VB | None | Mild | 67 | 2 | +| C12 | `6a2fab2` | V3 | **Severe** | Severe | 67 | **5** | +| C13 | `554af3b` | VB | None | Severe | 67 | 11 | +| C14 | `c710b55` | VB | None | Severe | 67 | 5 | +| C15 | `f13371c` | VB | None | Severe | 67 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C7 at 467 lines**. +The smallest drift-bearing diff is **C12 at 5 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C9 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C10 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C11 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C12 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C13 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C14 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C15 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C8 — V2, Mild + +`feat: let the app enumerate available bedrock models` + +- **Mechanism:** `bedrock_readonly_controlplane` +- **Path:** `EcsTaskRole -> bedrock:List/Get FoundationModel -> *` +- **Controls present:** ReadOnlyControlPlane +- **Component:** `EcsTaskRole` +- **Transitive reach:** enumerate model metadata +- **Template assertion:** policy with bedrock:ListFoundationModels + GetFoundationModel + +Read-only control-plane access; harmless enumeration. Mild. + +### C12 — V3, Severe + +`fix: grant full bedrock access to simplify feature work` + +- **Mechanism:** `bedrock_star` +- **Path:** `EcsTaskRole -> bedrock:* -> *` +- **Controls present:** none +- **Controls removed:** ReadOnlyControlPlane +- **Component:** `EcsTaskRole` +- **Transitive reach:** modify guardrails, disable invocation logging, manage models +- **Template assertion:** bedrock actions collapse to ['bedrock:*'] + +Control-plane authority: the app can weaken its own guardrails and logging. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no iam change +- **C3** `chore: add log level env to ingestion lambda` — small lambda diff +- **C4** `chore: raise ingestion lambda memory and timeout` — perf tuning +- **C5** `chore: raise cognito min password length` — security improvement decoy +- **C6** `chore: tighten alb health check interval` — target group only +- **C7** `chore: apply cost-allocation tags` — largest template diff +- **C9** `chore: add known-bad-inputs waf rule` — ALB only +- **C10** `chore: enable vpc flow logs` — security improvement decoy +- **C11** `chore: scale service to two tasks` — capacity only +- **C13** `chore: add kb bucket lifecycle rule` — retention only +- **C14** `chore: add vpc id stack output` — cosmetic output +- **C15** `docs: add bedrock runbook` — no template change while cumulative Severe + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- guardrail enforcement depends on the app calling ApplyGuardrail at C1 +- no separation between inference and Bedrock control-plane at C1 + +## Scoring + +- **Drift commits:** C8, C12 +- **Benign commits:** C2, C3, C4, C5, C6, C7, C9, C10, C11, C13, C14, C15 +- **False-positive probes:** C7, C9, C10 + +- **State-vs-event probes:** C13, C14, C15 + +AI control-plane privilege; Severe lets the app disable its own guardrails/logging. + +## Layout + +``` +G7/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark g7 --baseline /text-paste-service/v1/cdk --out G7 +python3 generators/synth.py --root G7 --workbench /tmp/wb +python3 generators/analyze.py --root G7 +python3 generators/verify.py --benchmark g7 --root G7 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/g7.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C10, C11, C12, C13, C14, C15, C2, C3, C4, C5, C6, C7, C8, C9 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/cdk/cdk.ts new file mode 100644 index 00000000..18320792 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/cdk/cdk.ts @@ -0,0 +1,614 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ListFoundationModels', 'bedrock:GetFoundationModel'], + resources: ['*'], + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/commit.json new file mode 100644 index 00000000..53a2d8c5 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C10", + "sha": "474fefd", + "full_sha": "474fefd8258f615a4f7afdb26c29bee91a14f083", + "parent": "80b8bed", + "message": "chore: enable vpc flow logs", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-06T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/diff.patch new file mode 100644 index 00000000..a9b9c05e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C10/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 8b916e0..1832079 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -83,6 +83,8 @@ export class AiChatbotStack extends cdk.Stack { + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + ++ vpc.addFlowLog('AppVpcFlowLog'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/cdk/cdk.ts new file mode 100644 index 00000000..93441a8c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/cdk/cdk.ts @@ -0,0 +1,614 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ListFoundationModels', 'bedrock:GetFoundationModel'], + resources: ['*'], + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/commit.json new file mode 100644 index 00000000..f295421a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C11", + "sha": "24c81d7", + "full_sha": "24c81d76c582d040505179804e333d58db68f8ff", + "parent": "474fefd", + "message": "chore: scale service to two tasks", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-13T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/diff.patch new file mode 100644 index 00000000..1fc56377 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C11/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 1832079..93441a8 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -421,7 +421,7 @@ export class AiChatbotStack extends cdk.Stack { + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, +- desiredCount: 1, // Start with one task ++ desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/cdk/cdk.ts new file mode 100644 index 00000000..308f5a09 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/cdk/cdk.ts @@ -0,0 +1,614 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:*'], + resources: ['*'], + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/commit.json new file mode 100644 index 00000000..54d42735 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C12", + "sha": "6a2fab2", + "full_sha": "6a2fab245222711a687e17df70228d8547af0601", + "parent": "24c81d7", + "message": "fix: grant full bedrock access to simplify feature work", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-05-20T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/diff.patch new file mode 100644 index 00000000..7e13dffa --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C12/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 93441a8..308f5a0 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -488,7 +488,7 @@ export class AiChatbotStack extends cdk.Stack { + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ +- actions: ['bedrock:ListFoundationModels', 'bedrock:GetFoundationModel'], ++ actions: ['bedrock:*'], + resources: ['*'], + })); \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/cdk/cdk.ts new file mode 100644 index 00000000..64250052 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/cdk/cdk.ts @@ -0,0 +1,620 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + knowledgeBaseBucket.addLifecycleRule({ + id: 'ExpireOldNoncurrentVersions', + noncurrentVersionExpiration: cdk.Duration.days(365), + enabled: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:*'], + resources: ['*'], + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/commit.json new file mode 100644 index 00000000..97283863 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C13", + "sha": "554af3b", + "full_sha": "554af3ba9058e412d2f713e55994a5458aeeda93", + "parent": "6a2fab2", + "message": "chore: add kb bucket lifecycle rule", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-27T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 6 ++++++\n 1 file changed, 6 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/diff.patch new file mode 100644 index 00000000..eb1dcaf1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C13/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 308f5a0..6425005 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -95,6 +95,12 @@ export class AiChatbotStack extends cdk.Stack { + enforceSSL: true, + }); + ++ knowledgeBaseBucket.addLifecycleRule({ ++ id: 'ExpireOldNoncurrentVersions', ++ noncurrentVersionExpiration: cdk.Duration.days(365), ++ enabled: true, ++ }); ++ + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/cdk/cdk.ts new file mode 100644 index 00000000..5df3d74a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/cdk/cdk.ts @@ -0,0 +1,621 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + knowledgeBaseBucket.addLifecycleRule({ + id: 'ExpireOldNoncurrentVersions', + noncurrentVersionExpiration: cdk.Duration.days(365), + enabled: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:*'], + resources: ['*'], + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'VpcIdOutput', { value: vpc.vpcId }); + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/commit.json new file mode 100644 index 00000000..c6bc00c9 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C14", + "sha": "c710b55", + "full_sha": "c710b55a4480bb8466981aa965ff694e04640b25", + "parent": "554af3b", + "message": "chore: add vpc id stack output", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-06-03T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/diff.patch new file mode 100644 index 00000000..7f111509 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C14/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 6425005..5df3d74 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -499,6 +499,7 @@ export class AiChatbotStack extends cdk.Stack { + })); + + // --- Outputs --- // ++ new cdk.CfnOutput(this, 'VpcIdOutput', { value: vpc.vpcId }); + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/cdk/cdk.ts new file mode 100644 index 00000000..5df3d74a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/cdk/cdk.ts @@ -0,0 +1,621 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + knowledgeBaseBucket.addLifecycleRule({ + id: 'ExpireOldNoncurrentVersions', + noncurrentVersionExpiration: cdk.Duration.days(365), + enabled: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:*'], + resources: ['*'], + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'VpcIdOutput', { value: vpc.vpcId }); + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/commit.json new file mode 100644 index 00000000..782cbf2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C15", + "sha": "f13371c", + "full_sha": "f13371cedfd70c7607efa4760c8896fac4f32d9a", + "parent": "c710b55", + "message": "docs: add bedrock runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-06-10T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C15/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..1c63332e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/commit.json new file mode 100644 index 00000000..ca03bac1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "78818a1", + "full_sha": "78818a19183164debd040b612f70d168f3735cb6", + "parent": "3a36f77", + "message": "chore: add log level env to ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/diff.patch new file mode 100644 index 00000000..b68e56bf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C3/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..1c63332 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -178,6 +178,7 @@ export class AiChatbotStack extends cdk.Stack { + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); ++ dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..2125336b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/commit.json new file mode 100644 index 00000000..07ac6ae0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "2f86549", + "full_sha": "2f86549d0505710a6fd2a938c55929cf43ef398a", + "parent": "78818a1", + "message": "chore: raise ingestion lambda memory and timeout", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/diff.patch new file mode 100644 index 00000000..ca91e69c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C4/diff.patch @@ -0,0 +1,15 @@ +diff --git a/cdk.ts b/cdk.ts +index 1c63332..2125336 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -115,8 +115,8 @@ export class AiChatbotStack extends cdk.Stack { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, +- timeout: cdk.Duration.minutes(5), +- memorySize: 512, ++ timeout: cdk.Duration.minutes(10), ++ memorySize: 1024, + }); + + // S3 event notification to trigger Lambda \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..b3495945 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/commit.json new file mode 100644 index 00000000..ecfda9ca --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "5057693", + "full_sha": "5057693519d966f0e787f1d31d3fae4bebef9437", + "parent": "2f86549", + "message": "chore: raise cognito min password length", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/diff.patch new file mode 100644 index 00000000..5442ca7e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C5/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 2125336..b349594 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -213,7 +213,7 @@ export class AiChatbotStack extends cdk.Stack { + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { +- minLength: 8, ++ minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..ace9f726 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/commit.json new file mode 100644 index 00000000..e8cf1669 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "0917e4c", + "full_sha": "0917e4cc02472d23e4a9d928378eea8779f8264c", + "parent": "5057693", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/diff.patch new file mode 100644 index 00000000..367ee1c5 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index b349594..ace9f72 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -413,7 +413,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..8c6c27c5 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/cdk/cdk.ts @@ -0,0 +1,591 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/commit.json new file mode 100644 index 00000000..f9f02a05 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "3c3cdd6", + "full_sha": "3c3cdd6ea3e1612d55456d848531a93900e6ec45", + "parent": "0917e4c", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/diff.patch new file mode 100644 index 00000000..3a244144 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C7/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index ace9f72..8c6c27c 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..ffe37ab3 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/cdk/cdk.ts @@ -0,0 +1,596 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ListFoundationModels', 'bedrock:GetFoundationModel'], + resources: ['*'], + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/commit.json new file mode 100644 index 00000000..9c11f162 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "f1c70e3", + "full_sha": "f1c70e30045cd8198a0a82cd71894e5de5192312", + "parent": "3c3cdd6", + "message": "feat: let the app enumerate available bedrock models", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/diff.patch new file mode 100644 index 00000000..8b53b8a2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C8/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 8c6c27c..ffe37ab 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -469,6 +469,11 @@ export class AiChatbotStack extends cdk.Stack { + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + ++ ecsTaskRole.addToPolicy(new iam.PolicyStatement({ ++ actions: ['bedrock:ListFoundationModels', 'bedrock:GetFoundationModel'], ++ resources: ['*'], ++ })); ++ + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/cdk/cdk.ts new file mode 100644 index 00000000..8b916e0b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/cdk/cdk.ts @@ -0,0 +1,612 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ListFoundationModels', 'bedrock:GetFoundationModel'], + resources: ['*'], + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/commit.json new file mode 100644 index 00000000..3e23ddd2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C9", + "sha": "80b8bed", + "full_sha": "80b8bed8cdc03aab3f7ed86b727eb28b9ffeee73", + "parent": "f1c70e3", + "message": "chore: add known-bad-inputs waf rule", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-29T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 16 ++++++++++++++++\n 1 file changed, 16 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/diff.patch new file mode 100644 index 00000000..f971db9d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/C9/diff.patch @@ -0,0 +1,27 @@ +diff --git a/cdk.ts b/cdk.ts +index ffe37ab..8b916e0 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -273,6 +273,22 @@ export class AiChatbotStack extends cdk.Stack { + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, ++ { ++ name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', ++ priority: 2, ++ statement: { ++ managedRuleGroupStatement: { ++ vendorName: 'AWS', ++ name: 'AWSManagedRulesKnownBadInputsRuleSet', ++ }, ++ }, ++ overrideAction: { none: {} }, ++ visibilityConfig: { ++ sampledRequestsEnabled: true, ++ cloudWatchMetricsEnabled: true, ++ metricName: 'AWSManagedRulesKnownBadInputsRuleSet', ++ }, ++ }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/provenance.json new file mode 100644 index 00000000..77c2367f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/commits/provenance.json @@ -0,0 +1,227 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C10": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "474fefd", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C11": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "24c81d7", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C12": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "6a2fab2", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C13": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "554af3b", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C14": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "c710b55", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C15": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "f13371c", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "78818a1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "2f86549", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "5057693", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "0917e4c", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3c3cdd6", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "f1c70e3", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C9": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "80b8bed", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/metadata.yaml new file mode 100644 index 00000000..b18ee000 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/metadata.yaml @@ -0,0 +1,196 @@ +benchmark: G7 +family: Privilege Drift +drift_name: Bedrock Control-Plane Wildcard +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "The ECS task role gains Bedrock list/get actions (C8, Mild), then bedrock:* (C12, Severe)\ + \ \u2014 control-plane authority over models, guardrails and invocation logging." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'Bedrock actions on the task role: list/get -> bedrock:*.' +commit_count: 15 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no iam change + C3: + sha: 78818a1 + label: VB + message: 'chore: add log level env to ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: small lambda diff + C4: + sha: 2f86549 + label: VB + message: 'chore: raise ingestion lambda memory and timeout' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: perf tuning + C5: + sha: '5057693' + label: VB + message: 'chore: raise cognito min password length' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: security improvement decoy + C6: + sha: 0917e4c + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: target group only + C7: + sha: 3c3cdd6 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: largest template diff + C8: + sha: f1c70e3 + label: V2 + message: 'feat: let the app enumerate available bedrock models' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: G1 + operation: add + mechanism: bedrock_readonly_controlplane + path: EcsTaskRole -> bedrock:List/Get FoundationModel -> * + controls_present: + - ReadOnlyControlPlane + controls_absent: [] + component: EcsTaskRole + transitive_reach: + - enumerate model metadata + construct_ids: + - EcsTaskRole + template_assertion: policy with bedrock:ListFoundationModels + GetFoundationModel + note: Read-only control-plane access; harmless enumeration. Mild. + C9: + sha: 80b8bed + label: VB + message: 'chore: add known-bad-inputs waf rule' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: ALB only + C10: + sha: 474fefd + label: VB + message: 'chore: enable vpc flow logs' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: security improvement decoy + C11: + sha: 24c81d7 + label: VB + message: 'chore: scale service to two tasks' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: capacity only + C12: + sha: 6a2fab2 + label: V3 + message: 'fix: grant full bedrock access to simplify feature work' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: G1 + operation: mutate + mechanism: bedrock_star + path: EcsTaskRole -> bedrock:* -> * + controls_present: [] + controls_removed: + - ReadOnlyControlPlane + component: EcsTaskRole + transitive_reach: + - modify guardrails, disable invocation logging, manage models + construct_ids: + - EcsTaskRole + template_assertion: bedrock actions collapse to ['bedrock:*'] + note: 'Control-plane authority: the app can weaken its own guardrails and logging.' + C13: + sha: 554af3b + label: VB + message: 'chore: add kb bucket lifecycle rule' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: retention only + C14: + sha: c710b55 + label: VB + message: 'chore: add vpc id stack output' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: cosmetic output + C15: + sha: f13371c + label: VB + message: 'docs: add bedrock runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe +baseline_known_weaknesses: +- guardrail enforcement depends on the app calling ApplyGuardrail at C1 +- no separation between inference and Bedrock control-plane at C1 +scoring: + drift_commits: + - C8 + - C12 + benign_commits: + - C2 + - C3 + - C4 + - C5 + - C6 + - C7 + - C9 + - C10 + - C11 + - C13 + - C14 + - C15 + false_positive_probes: + - C7 + - C9 + - C10 + state_vs_event_probes: + - C13 + - C14 + - C15 + severity_axis: Bedrock action breadth (read-only control-plane -> bedrock:*) + note: AI control-plane privilege; Severe lets the app disable its own guardrails/logging. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/repository new file mode 160000 index 00000000..f13371ce --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/repository @@ -0,0 +1 @@ +Subproject commit f13371cedfd70c7607efa4760c8896fac4f32d9a diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/template_analysis.json new file mode 100644 index 00000000..f9cfd401 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/G7/template_analysis.json @@ -0,0 +1,472 @@ +{ + "benchmark": "G7", + "drift_name": "Bedrock Control-Plane Wildcard", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "78818a1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 1, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "2f86549", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 4, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "5057693", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "0917e4c", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "3c3cdd6", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "f1c70e3", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 8, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C9": { + "sha": "80b8bed", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 18, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C10": { + "sha": "474fefd", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 139, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C11": { + "sha": "24c81d7", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C12": { + "sha": "6a2fab2", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 5, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [ + "EcsTaskRoleDefaultPolicy50882C77:bedrock:*" + ], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C13": { + "sha": "554af3b", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 11, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [ + "EcsTaskRoleDefaultPolicy50882C77:bedrock:*" + ], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C14": { + "sha": "c710b55", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 5, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [ + "EcsTaskRoleDefaultPolicy50882C77:bedrock:*" + ], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C15": { + "sha": "f13371c", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", + "EcsTaskRoleDefaultPolicy50882C77" + ], + "wildcard_action_statements": [ + "EcsTaskRoleDefaultPolicy50882C77:bedrock:*" + ], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/PULL_REQUEST.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/PULL_REQUEST.md new file mode 100644 index 00000000..7e7536a6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/PULL_REQUEST.md @@ -0,0 +1,150 @@ +# CDK Drift Benchmark Suite — 32 benchmarks across 5 families + +## Summary + +Adds `benchmarks-chatbot/` — 32 independent parallel evolutions of the `aws-ai-chat-bot` +CDK stack, one per drift scenario. Every benchmark is a real git repository with real +commits (4 to 15 each, 4 minimum) and a per-commit synthesized CloudFormation template. +All 32 share commit C1 = `9c19d32` byte-for-byte; divergence starts at C2. + +Companion to the 25-benchmark [`benchmarks/`](../benchmarks) (text-paste), 30-benchmark +[`benchmarks-ecs/`](../benchmarks-ecs) (containerized-microservices) and 28-benchmark +[`benchmarks-support/`](../benchmarks-support) (support-ticketing) suites. This is the +first **generative-AI** baseline: a Bedrock RAG chatbot with an OpenSearch Serverless +vector store, Bedrock (Claude + Titan) behind a PII guardrail, Cognito, WAF, ECS Fargate +on a public ALB, an S3 knowledge base feeding a Python ingestion Lambda, DynamoDB chat +history, and Bedrock/DynamoDB VPC endpoints. + +**Total: 32 benchmarks, 304 commits, 78 MB, 100% verified and typechecked.** Four suites +now total 25 + 30 + 28 + 32 = 115 benchmarks. + +## Drift families + +Each family holds severity to a single axis and every benchmark labels exactly two +commits — V2 (Mild) and V3 (Severe) — with the rest benign. + +| Family | Axis | Benchmarks | +|---|---|---| +| **R — Architectural Reachability** | internal compute/data becomes reachable, then loses auth | R1 vector store public, R2 WAF detached, R3 ECS SG bypass, R4 public ingestion upload, R5 ingestion Function URL, R6 Bedrock endpoint reach | +| **G — Privilege** | IAM breadth (actions × resources) | G1 Bedrock model→*, G2 aoss wildcard, G3 AssumeRole, G4 Administrator, G5 KB write escalation, G6 DynamoDB wildcard, G7 `bedrock:*` | +| **E — Exposure** | existing surface loses restrictions | E1 KB BlockPublicAccess, E2 aoss data policy, E3 Cognito open callback, E4 KB bucket policy, E5 Bedrock endpoint policy, E6 WAF rule neutered | +| **D — Dependency / Integration** | third-party enters the prompt/data path | D1 OpenAI bypass, D2 Pinecone primary, D3 trace full-IO, D4 tool webhook sig, D5 external embedder, D6 model gateway | +| **T — Trust Boundary** | privilege crosses account/env boundary | T1 VPC peering, T2 cross-account IAM, T3 vector foreign account, T4 cross-account model, T5 Cognito federation, T6 secret cross-account, T7 event bus cross-account | + +AI-native cross-family pairs are scoreable — **G1↔T4** (Bedrock model access: in-account +wildcard vs cross-account/region), **R1↔E2↔T3** (vector store: network vs data-policy vs +foreign account), **G4↔T2** (`AdministratorAccess`: internal vs cross-account), **D1↔D5↔D6** +(external model/embedder/gateway bypassing the Bedrock guardrail). + +## Commit log structure + +Every benchmark's `repository/` is a genuine git repo. `git log`, `diff`, `show`, +`bisect` all work (R1 shown): + +``` +e99cee5 docs: on-call contact note # VB benign +90d2d91 chore: extend gitignore # VB benign +f0fcc24 docs: add ingestion runbook # VB benign +6221c3a fix: open vector collection for external partner dashboards # V3 Severe (R1) +388da16 chore: apply cost-allocation tags # VB benign +455d381 feat: allow admin tooling vpc to reach the vector collection # V2 Mild +e04546e chore: add log level env to ingestion lambda # VB benign +4bcd5c2 chore: add cloudwatch error alarm on ingestion lambda # VB benign +9c19d32 feat: bedrock rag chatbot platform baseline # V1 (shared) +``` + +Author, date and identity are pinned so SHAs are reproducible. Every benchmark's C1 is +`9c19d32`; divergence starts at C2. + +Commit-count distribution (no drift-position collision within a length): + +``` +count: 4 5 6 7 8 9 10 11 12 13 14 15 +benchmark: E1 D4 R6 E5 G6 R2 D2 E3 G2 D5 R4 T1 E2 R1 T2 D1 G1 T5 D3 R3 T3 D6 G5 T6 E4 R5 T7 E6 G3 G7 T4 + G4 +``` + +## Per-benchmark layout + +Each `benchmarks-chatbot//` mirrors the source dataset's `/cdk/` folder shape: + +``` +R1/ +├── repository/ # real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..CN/ +│ │ ├── cdk/ # same layout as aws-ai-chat-bot/cdk/ +│ │ │ ├── cdk.ts # the stack at this commit +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda_processor/ # the real Python ingestion handler +│ │ │ └── cdk.out/ # synthesized template, tree.json, manifests +│ │ ├── commit.json # sha, parent, author, label, severities, diffstat +│ │ └── diff.patch # diff against parent (empty for C1) +│ └── provenance.json # synth version and resource count per commit +├── metadata.yaml # ground truth — generated from the benchmark module (researcher use for validation) +├── template_analysis.json # measured diffs and control state per commit (researcher use for validation) +└── README.md # generated spec (researcher use for validation) +``` + +Tooling written against `aws-ai-chat-bot/cdk/cdk.out/*.template.json` works against +`benchmarks-chatbot/R1/commits/C5/cdk/cdk.out/*.template.json` with only the root path +changed. Resource counts run 61 (baseline) → 69. + +## metadata.yaml — the answer key + +Deliberately outside `repository/` so it can't leak into anything a detector reads. +Per commit: `sha`, `label` (V1/V2/V3/VB), `commit_severity`, `cumulative_severity`, and +for drift commits a `ground_truth` block with mechanism, path, controls +present/absent/removed, transitive reach, and a `template_assertion` string. + +Two severity fields deliberately: `commit_severity` = drift introduced by that commit; +`cumulative_severity` = standing risk at that commit. The commits after a Severe drift sit +at non-zero cumulative severity while introducing nothing — a false-positive report there +is wrong, and so is "all clear". + +## Suite property worth naming + +Template diff size is deliberately anti-correlated with severity across the suite. Benign +cost-allocation tag commits propagate onto every taggable resource and produce the largest +diffs in every benchmark (437–519 lines); severe commits typically flip one field: + +| | Severe | benign (biggest) | +|---|---|---| +| T2 | 2 lines (cross-account role → Administrator) | 517 lines | +| G3 | 2 lines (`sts:AssumeRole` → `*`) | 503 lines | +| T5 | 2 lines (Cognito issuer → open broker) | 503 lines | +| R1 | 2 lines (`AllowFromPublic` false → true) | 467 lines | +| D4 | 2 lines (`TOOL_VERIFY_SIGNATURE` true → false) | 467 lines | +| G4 | 2 lines (PowerUser → Administrator) | 437 lines | + +Any detector triaging by diff magnitude scores worse than chance. + +## Quadruple-baseline oracle (free ground truth) + +The source dataset's knowledge-base bucket uses `BlockPublicAccess.BLOCK_ALL`; **E1** +drifts it to all-false — the same mechanism as `benchmarks/E5` (paste), `benchmarks-ecs/E3` +and `benchmarks-support/E1`. Detector performance on synthetic vs. real curated drift for +one mechanism is now comparable across **four** different stacks. + +## Known limitation + +The baseline ships no `package.json`, so no `aws-cdk-lib` version is declared; all +templates were synthesized with 2.150.0 and commits flag `version_match: false` in +`provenance.json` (declaring the 2.100.0 default supplied at import while synthesized with +2.150.0). Within a benchmark every template comes from one library version, so cross-commit +diffs are clean. One always-truthy baseline expression, `ec2.Port.tcp(3000 || 3000)`, fails +tsc (TS2872) and is normalized to `ec2.Port.tcp(3000)` — a no-op since `3000 || 3000` is +`3000`. Because raw `ts-node` synth does not read `cdk.json`, the required context (account, +region, `dockerImageName`, and the feature flags `enablePiiRedaction`/`enableChatHistory`, +both true) is injected via `synth_context.json`, so the guardrail, Cognito and DynamoDB +chat history all exist at C1. + +## Not included + +`threat-model.json`, `canonical_ir.json`, the `7.*` graph IRs, `architecture.png` and +`diagram.py` from the source dataset are absent — they come from tooling outside this +generator. A stale threat model copied forward would falsely assert "nothing changed" at +exactly the commits where the guardrail is bypassed or the vector store is opened, which is +worse than having none. Run your own pipeline over `commits/C*/cdk/` to produce them. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/README.md new file mode 100644 index 00000000..5c6171e4 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/README.md @@ -0,0 +1,150 @@ +# R1 — Vector Store Public Exposure + +**Family:** Architectural Reachability Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The OpenSearch Serverless vector collection network policy adds a second VPC endpoint (C4, Mild), then flips AllowFromPublic to true (C6, Severe) — the RAG knowledge base is reachable from the public internet. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +OpenSearch Serverless network policy: private VPCE-only -> public. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `4bcd5c2` | VB | None | None | 63 | 32 | +| C3 | `e04546e` | VB | None | None | 63 | 1 | +| C4 | `455d381` | V2 | **Mild** | Mild | 63 | **2** | +| C5 | `388da16` | VB | None | Mild | 63 | 467 | +| C6 | `6221c3a` | V3 | **Severe** | Severe | 63 | **2** | +| C7 | `f0fcc24` | VB | None | Severe | 63 | **0** | +| C8 | `90d2d91` | VB | None | Severe | 63 | **0** | +| C9 | `e99cee5` | VB | None | Severe | 63 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C5 at 467 lines**. +The smallest drift-bearing diff is **C4 at 2 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C9 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C4 — V2, Mild + +`feat: allow admin tooling vpc to reach the vector collection` + +- **Mechanism:** `aoss_network_extra_vpce` +- **Path:** `admin-tooling VPC -> aoss:APIAccessAll -> VectorCollection` +- **Controls present:** PrivateVpceOnly, AllowFromPublicFalse +- **Component:** `VectorCollection` +- **Transitive reach:** one additional internal network +- **Template assertion:** SecurityPolicy network rule SourceVPCEs gains a second endpoint + +Reach broadens to another private network; still not public. Mild. + +### C6 — V3, Severe + +`fix: open vector collection for external partner dashboards` + +- **Mechanism:** `aoss_public_access` +- **Path:** `internet -> VectorCollection endpoint` +- **Controls present:** none +- **Controls removed:** AllowFromPublicFalse, PrivateVpceOnly +- **Component:** `VectorCollection` +- **Transitive reach:** the entire RAG knowledge base, from anywhere +- **Template assertion:** network SecurityPolicy AllowFromPublic flips false -> true + +The vector store behind every answer is now internet-reachable. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no network-policy change +- **C3** `chore: add log level env to ingestion lambda` — small lambda diff +- **C5** `chore: apply cost-allocation tags` — largest template diff +- **C7** `docs: add ingestion runbook` — no template change while cumulative Severe +- **C8** `chore: extend gitignore` — zero template diff +- **C9** `docs: on-call contact note` — docs-only + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- albSecurityGroup 0.0.0.0/0 on 80/443 at C1 +- aoss data-access grants APIAccessAll (broad) at C1 + +## Scoring + +- **Drift commits:** C4, C6 +- **Benign commits:** C2, C3, C5, C7, C8, C9 +- **False-positive probes:** C5 + +- **State-vs-event probes:** C7, C8, C9 + +New AI-native vector-store exposure axis; pairs with E2/T3 (data-access policy). + +## Layout + +``` +R1/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark r1 --baseline /text-paste-service/v1/cdk --out R1 +python3 generators/synth.py --root R1 --workbench /tmp/wb +python3 generators/analyze.py --root R1 +python3 generators/verify.py --benchmark r1 --root R1 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/r1.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C2, C3, C4, C5, C6, C7, C8, C9 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/commit.json new file mode 100644 index 00000000..a19d9236 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "4bcd5c2", + "full_sha": "4bcd5c206fe43b79996a2155c5647455c5dbaf42", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..1c63332e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/commit.json new file mode 100644 index 00000000..532267d4 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "e04546e", + "full_sha": "e04546e3572cedadc9740ce1b5d528a331be5317", + "parent": "4bcd5c2", + "message": "chore: add log level env to ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/diff.patch new file mode 100644 index 00000000..b68e56bf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C3/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..1c63332 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -178,6 +178,7 @@ export class AiChatbotStack extends cdk.Stack { + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); ++ dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..e10f3e15 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId, 'vpce-0admintooling00000'], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/commit.json new file mode 100644 index 00000000..410dbd4e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "455d381", + "full_sha": "455d3818116bf05561ac6c6798aaa14aff838d1a", + "parent": "e04546e", + "message": "feat: allow admin tooling vpc to reach the vector collection", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/diff.patch new file mode 100644 index 00000000..466ab496 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C4/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 1c63332..e10f3e1 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -140,7 +140,7 @@ export class AiChatbotStack extends cdk.Stack { + }, + ], + AllowFromPublic: false, // Keep it private +- SourceVPCEs: [vpc.vpcId], ++ SourceVPCEs: [vpc.vpcId, 'vpce-0admintooling00000'], + }, + ]; \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..ca356042 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/cdk/cdk.ts @@ -0,0 +1,591 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId, 'vpce-0admintooling00000'], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/commit.json new file mode 100644 index 00000000..136b6975 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "388da16", + "full_sha": "388da16d70441de252853e8060ac812f7c460f57", + "parent": "455d381", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/diff.patch new file mode 100644 index 00000000..d5fc14a1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C5/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index e10f3e1..ca35604 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..bbaadcdd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/cdk/cdk.ts @@ -0,0 +1,591 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: true, // Opened for external partner dashboards + SourceVPCEs: [vpc.vpcId, 'vpce-0admintooling00000'], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/commit.json new file mode 100644 index 00000000..48a7d144 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "6221c3a", + "full_sha": "6221c3a32a99d49e638c64529556a8773d96b87c", + "parent": "388da16", + "message": "fix: open vector collection for external partner dashboards", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/diff.patch new file mode 100644 index 00000000..58cb1513 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index ca35604..bbaadcd 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -144,7 +144,7 @@ export class AiChatbotStack extends cdk.Stack { + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], +- AllowFromPublic: false, // Keep it private ++ AllowFromPublic: true, // Opened for external partner dashboards + SourceVPCEs: [vpc.vpcId, 'vpce-0admintooling00000'], + }, + ]; \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..bbaadcdd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/cdk/cdk.ts @@ -0,0 +1,591 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: true, // Opened for external partner dashboards + SourceVPCEs: [vpc.vpcId, 'vpce-0admintooling00000'], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/commit.json new file mode 100644 index 00000000..7060a72d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "f0fcc24", + "full_sha": "f0fcc2424d8c9aa63b2ca4e9f0840cd920f8cc8c", + "parent": "6221c3a", + "message": "docs: add ingestion runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C7/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..bbaadcdd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/cdk/cdk.ts @@ -0,0 +1,591 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: true, // Opened for external partner dashboards + SourceVPCEs: [vpc.vpcId, 'vpce-0admintooling00000'], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/commit.json new file mode 100644 index 00000000..5153c7d1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "90d2d91", + "full_sha": "90d2d91e63ba0996c3bad1ea186208ad049f8cff", + "parent": "f0fcc24", + "message": "chore: extend gitignore", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + ".gitignore" + ], + "diffstat": ".gitignore | 3 +++\n 1 file changed, 3 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/diff.patch new file mode 100644 index 00000000..81ec2928 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C8/diff.patch @@ -0,0 +1,11 @@ +diff --git a/.gitignore b/.gitignore +index a3dd4ae..a4c0727 100644 +--- a/.gitignore ++++ b/.gitignore +@@ -1,3 +1,6 @@ + node_modules/ + cdk.out/ + *.log ++.DS_Store ++coverage/ ++*.tsbuildinfo \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/cdk/cdk.ts new file mode 100644 index 00000000..bbaadcdd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/cdk/cdk.ts @@ -0,0 +1,591 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: true, // Opened for external partner dashboards + SourceVPCEs: [vpc.vpcId, 'vpce-0admintooling00000'], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/commit.json new file mode 100644 index 00000000..e7afe472 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C9", + "sha": "e99cee5", + "full_sha": "e99cee51e09688c5594e18f78896883447d95b11", + "parent": "90d2d91", + "message": "docs: on-call contact note", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-29T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 4 ++++\n 1 file changed, 4 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/diff.patch new file mode 100644 index 00000000..a3c07d85 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/C9/diff.patch @@ -0,0 +1,12 @@ +diff --git a/README.md b/README.md +index c7133f7..5516cf8 100644 +--- a/README.md ++++ b/README.md +@@ -10,3 +10,7 @@ DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts P + + - Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. + - Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. ++ ++## Contacts ++ ++- On-call: ml-platform@example.com \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/provenance.json new file mode 100644 index 00000000..3d20ba0b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/commits/provenance.json @@ -0,0 +1,137 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "4bcd5c2", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "e04546e", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "455d381", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "388da16", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "6221c3a", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "f0fcc24", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "90d2d91", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C9": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "e99cee5", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/metadata.yaml new file mode 100644 index 00000000..41b56f23 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/metadata.yaml @@ -0,0 +1,145 @@ +benchmark: R1 +family: Architectural Reachability Drift +drift_name: Vector Store Public Exposure +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "The OpenSearch Serverless vector collection network policy adds a second VPC endpoint\ + \ (C4, Mild), then flips AllowFromPublic to true (C6, Severe) \u2014 the RAG knowledge base is\ + \ reachable from the public internet." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'OpenSearch Serverless network policy: private VPCE-only -> public.' +commit_count: 9 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 4bcd5c2 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no network-policy change + C3: + sha: e04546e + label: VB + message: 'chore: add log level env to ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: small lambda diff + C4: + sha: 455d381 + label: V2 + message: 'feat: allow admin tooling vpc to reach the vector collection' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: R1 + operation: add + mechanism: aoss_network_extra_vpce + path: admin-tooling VPC -> aoss:APIAccessAll -> VectorCollection + controls_present: + - PrivateVpceOnly + - AllowFromPublicFalse + controls_absent: [] + component: VectorCollection + transitive_reach: + - one additional internal network + construct_ids: + - VectorCollection + - OpenSearchNetworkPolicy + template_assertion: SecurityPolicy network rule SourceVPCEs gains a second endpoint + note: Reach broadens to another private network; still not public. Mild. + C5: + sha: 388da16 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C6: + sha: 6221c3a + label: V3 + message: 'fix: open vector collection for external partner dashboards' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: R1 + operation: mutate + mechanism: aoss_public_access + path: internet -> VectorCollection endpoint + controls_present: [] + controls_removed: + - AllowFromPublicFalse + - PrivateVpceOnly + component: VectorCollection + transitive_reach: + - the entire RAG knowledge base, from anywhere + construct_ids: + - VectorCollection + - OpenSearchNetworkPolicy + template_assertion: network SecurityPolicy AllowFromPublic flips false -> true + note: The vector store behind every answer is now internet-reachable. + C7: + sha: f0fcc24 + label: VB + message: 'docs: add ingestion runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe + C8: + sha: 90d2d91 + label: VB + message: 'chore: extend gitignore' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: zero template diff + C9: + sha: e99cee5 + label: VB + message: 'docs: on-call contact note' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: docs-only +baseline_known_weaknesses: +- albSecurityGroup 0.0.0.0/0 on 80/443 at C1 +- aoss data-access grants APIAccessAll (broad) at C1 +scoring: + drift_commits: + - C4 + - C6 + benign_commits: + - C2 + - C3 + - C5 + - C7 + - C8 + - C9 + false_positive_probes: + - C5 + state_vs_event_probes: + - C7 + - C8 + - C9 + severity_axis: OpenSearch Serverless network policy Principal breadth (VPCE -> public) + note: New AI-native vector-store exposure axis; pairs with E2/T3 (data-access policy). diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/repository new file mode 160000 index 00000000..e99cee51 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/repository @@ -0,0 +1 @@ +Subproject commit e99cee51e09688c5594e18f78896883447d95b11 diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/template_analysis.json new file mode 100644 index 00000000..e0eb49f9 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R1/template_analysis.json @@ -0,0 +1,276 @@ +{ + "benchmark": "R1", + "drift_name": "Vector Store Public Exposure", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "4bcd5c2", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "e04546e", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 1, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "455d381", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "388da16", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "6221c3a", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "f0fcc24", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "90d2d91", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C9": { + "sha": "e99cee5", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/README.md new file mode 100644 index 00000000..9e5f8d7a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/README.md @@ -0,0 +1,142 @@ +# R2 — WAF Removed from ALB + +**Family:** Architectural Reachability Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The WebACL drops the IP-reputation managed rule group (C3, Mild), then the WebACLAssociation is removed entirely (C5, Severe) so the public ALB is no longer protected by WAF. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +WAF coverage on the public ALB: full -> reduced rules -> unassociated. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `4bcd5c2` | VB | None | None | 63 | 32 | +| C3 | `9cdca8b` | V2 | **Mild** | Mild | 63 | **18** | +| C4 | `6b66ff7` | VB | None | Mild | 63 | 467 | +| C5 | `49c3649` | V3 | **Severe** | Severe | 62 | **14** | +| C6 | `2220125` | VB | None | Severe | 62 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C4 at 467 lines**. +The smallest drift-bearing diff is **C5 at 14 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C3 — V2, Mild + +`fix: drop ip-reputation rule group to cut waf false positives` + +- **Mechanism:** `waf_rule_group_removed` +- **Path:** `internet -> (WAF: CommonRuleSet only) -> ALB` +- **Controls present:** WebACLAssociation, CommonRuleSet +- **Controls removed:** AmazonIpReputationList +- **Component:** `WebACL` +- **Transitive reach:** known-bad-IP traffic no longer filtered +- **Template assertion:** WebACL Rules loses the AmazonIpReputationList managed group + +Protection weakened but WAF still associated. Mild. + +### C5 — V3, Severe + +`fix: detach webacl from alb during latency incident` + +- **Mechanism:** `waf_disassociated` +- **Path:** `internet -> ALB (no WAF)` +- **Controls present:** none +- **Controls removed:** WebACLAssociation +- **Component:** `ApplicationLoadBalancer` +- **Transitive reach:** all L7 traffic reaches the app unfiltered +- **Template assertion:** AWS::WAFv2::WebACLAssociation resource no longer present + +The WebACL still exists but protects nothing; the ALB is unshielded. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no WAF change +- **C4** `chore: apply cost-allocation tags` — largest template diff +- **C6** `docs: add ingestion runbook` — no template change while cumulative Severe + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- albSecurityGroup 0.0.0.0/0 on 80/443 at C1 +- ALB listens on plain HTTP 80 at C1 + +## Scoring + +- **Drift commits:** C3, C5 +- **Benign commits:** C2, C4, C6 +- **False-positive probes:** C4 + +- **State-vs-event probes:** C6 + +Protection-control removal; the WebACL resource lingers unassociated at V3. + +## Layout + +``` +R2/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark r2 --baseline /text-paste-service/v1/cdk --out R2 +python3 generators/synth.py --root R2 --workbench /tmp/wb +python3 generators/analyze.py --root R2 +python3 generators/verify.py --benchmark r2 --root R2 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/r2.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C2, C3, C4, C5, C6 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/commit.json new file mode 100644 index 00000000..a19d9236 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "4bcd5c2", + "full_sha": "4bcd5c206fe43b79996a2155c5647455c5dbaf42", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..d5971377 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/cdk/cdk.ts @@ -0,0 +1,569 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/commit.json new file mode 100644 index 00000000..621631aa --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "9cdca8b", + "full_sha": "9cdca8bef0d8f8310ee175b80814e61869583321", + "parent": "4bcd5c2", + "message": "fix: drop ip-reputation rule group to cut waf false positives", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 16 ----------------\n 1 file changed, 16 deletions(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/diff.patch new file mode 100644 index 00000000..c1421e26 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C3/diff.patch @@ -0,0 +1,26 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..d597137 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -267,22 +267,6 @@ export class AiChatbotStack extends cdk.Stack { + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, +- { +- name: 'AWS-AWSManagedRulesAmazonIpReputationList', +- priority: 1, +- statement: { +- managedRuleGroupStatement: { +- vendorName: 'AWS', +- name: 'AWSManagedRulesAmazonIpReputationList', +- }, +- }, +- overrideAction: { none: {} }, +- visibilityConfig: { +- sampledRequestsEnabled: true, +- cloudWatchMetricsEnabled: true, +- metricName: 'AWSManagedRulesAmazonIpReputationList', +- }, +- }, + ], + }); \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..d5cfdfd0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/cdk/cdk.ts @@ -0,0 +1,574 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/commit.json new file mode 100644 index 00000000..a2e13550 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "6b66ff7", + "full_sha": "6b66ff733b2009b64018393a03c9ccb3a7321e84", + "parent": "9cdca8b", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/diff.patch new file mode 100644 index 00000000..ff72ca8c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C4/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index d597137..d5cfdfd 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..95539f4c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/cdk/cdk.ts @@ -0,0 +1,569 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/commit.json new file mode 100644 index 00000000..098672cc --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "49c3649", + "full_sha": "49c3649d8c01adc0ebd42996e8529feca1c14e26", + "parent": "6b66ff7", + "message": "fix: detach webacl from alb during latency incident", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 -----\n 1 file changed, 5 deletions(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/diff.patch new file mode 100644 index 00000000..eab24a57 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C5/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index d5cfdfd..95539f4 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -408,11 +408,6 @@ export class AiChatbotStack extends cdk.Stack { + healthyHttpCodes: "200-299", + }); + +- // Associate WAF with ALB +- new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { +- resourceArn: fargateService.loadBalancer.loadBalancerArn, +- webAclArn: webAcl.attrArn, +- }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..95539f4c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/cdk/cdk.ts @@ -0,0 +1,569 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/commit.json new file mode 100644 index 00000000..1ea41f6e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "2220125", + "full_sha": "22201250cbb6bcd94f20c5c3be1156fe2fb7c8c3", + "parent": "49c3649", + "message": "docs: add ingestion runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/provenance.json new file mode 100644 index 00000000..3a53b1fd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/commits/provenance.json @@ -0,0 +1,92 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "4bcd5c2", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "9cdca8b", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "6b66ff7", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 62, + "sha": "49c3649", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 62, + "sha": "2220125", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/metadata.yaml new file mode 100644 index 00000000..ed127f49 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/metadata.yaml @@ -0,0 +1,114 @@ +benchmark: R2 +family: Architectural Reachability Drift +drift_name: WAF Removed from ALB +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: The WebACL drops the IP-reputation managed rule group (C3, Mild), then the WebACLAssociation + is removed entirely (C5, Severe) so the public ALB is no longer protected by WAF. +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'WAF coverage on the public ALB: full -> reduced rules -> unassociated.' +commit_count: 6 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 4bcd5c2 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no WAF change + C3: + sha: 9cdca8b + label: V2 + message: 'fix: drop ip-reputation rule group to cut waf false positives' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: R1 + operation: mutate + mechanism: waf_rule_group_removed + path: 'internet -> (WAF: CommonRuleSet only) -> ALB' + controls_present: + - WebACLAssociation + - CommonRuleSet + controls_removed: + - AmazonIpReputationList + component: WebACL + transitive_reach: + - known-bad-IP traffic no longer filtered + construct_ids: + - WebACL + template_assertion: WebACL Rules loses the AmazonIpReputationList managed group + note: Protection weakened but WAF still associated. Mild. + C4: + sha: 6b66ff7 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C5: + sha: 49c3649 + label: V3 + message: 'fix: detach webacl from alb during latency incident' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: R1 + operation: remove + mechanism: waf_disassociated + path: internet -> ALB (no WAF) + controls_present: [] + controls_removed: + - WebACLAssociation + component: ApplicationLoadBalancer + transitive_reach: + - all L7 traffic reaches the app unfiltered + construct_ids: + - WebACLAssociation + - AppService + template_assertion: AWS::WAFv2::WebACLAssociation resource no longer present + note: The WebACL still exists but protects nothing; the ALB is unshielded. + C6: + sha: '2220125' + label: VB + message: 'docs: add ingestion runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe +baseline_known_weaknesses: +- albSecurityGroup 0.0.0.0/0 on 80/443 at C1 +- ALB listens on plain HTTP 80 at C1 +scoring: + drift_commits: + - C3 + - C5 + benign_commits: + - C2 + - C4 + - C6 + false_positive_probes: + - C4 + state_vs_event_probes: + - C6 + severity_axis: WAF coverage on the public ALB + note: Protection-control removal; the WebACL resource lingers unassociated at V3. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/repository new file mode 160000 index 00000000..22201250 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/repository @@ -0,0 +1 @@ +Subproject commit 22201250cbb6bcd94f20c5c3be1156fe2fb7c8c3 diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/template_analysis.json new file mode 100644 index 00000000..ba467d81 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R2/template_analysis.json @@ -0,0 +1,186 @@ +{ + "benchmark": "R2", + "drift_name": "WAF Removed from ALB", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "4bcd5c2", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "9cdca8b", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 18, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "6b66ff7", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "49c3649", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 62, + "template_diff_lines": 14, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "2220125", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 62, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/README.md new file mode 100644 index 00000000..72f85c1e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/README.md @@ -0,0 +1,156 @@ +# R3 — ECS Task SG Bypasses ALB + +**Family:** Architectural Reachability Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The Fargate task security group opens the container port to the VPC CIDR (C5, Mild), then to 0.0.0.0/0 (C8, Severe) — clients reach the container directly, bypassing the ALB and its WAF. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +ecsSecurityGroup ingress on tcp/3000: none -> VPC CIDR -> 0.0.0.0/0. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `78818a1` | VB | None | None | 63 | 1 | +| C4 | `5bde6bd` | VB | None | None | 63 | 2 | +| C5 | `91b68fc` | V2 | **Mild** | Mild | 63 | **14** | +| C6 | `598a2da` | VB | None | Mild | 63 | 2 | +| C7 | `1612795` | VB | None | Mild | 63 | 467 | +| C8 | `be29565` | V3 | **Severe** | Severe | 63 | **9** | +| C9 | `79a9c85` | VB | None | Severe | 63 | 18 | +| C10 | `ed366cb` | VB | None | Severe | 63 | **0** | +| C11 | `3dba767` | VB | None | Severe | 63 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C7 at 467 lines**. +The smallest drift-bearing diff is **C8 at 9 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C9 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C10 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C11 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C5 — V2, Mild + +`feat: allow in-vpc debug access to task container port` + +- **Mechanism:** `ecs_sg_vpc_ingress` +- **Path:** `VPC clients -> tcp/3000 -> Fargate task` +- **Controls present:** VpcCidrScoped +- **Component:** `EcsSecurityGroup` +- **Transitive reach:** the app container, from any VPC host +- **Template assertion:** SecurityGroupIngress tcp/3000 from the VPC CIDR appears + +Direct-to-task path opens but stays inside the VPC. Mild. + +### C8 — V3, Severe + +`fix: expose task port for external uptime checks` + +- **Mechanism:** `ecs_sg_public_ingress` +- **Path:** `internet -> tcp/3000 -> Fargate task (ALB + WAF bypassed)` +- **Controls present:** none +- **Controls removed:** VpcCidrScoped +- **Component:** `EcsSecurityGroup` +- **Transitive reach:** the app container, from anywhere, unfiltered +- **Template assertion:** SecurityGroupIngress tcp/3000 CidrIp 0.0.0.0/0 + +Direct internet reach to the container; ALB auth and WAF no longer in path. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no SG change +- **C3** `chore: add log level env to ingestion lambda` — small lambda diff +- **C4** `chore: tighten alb health check interval` — target group only +- **C6** `chore: raise cognito min password length` — security improvement decoy +- **C7** `chore: apply cost-allocation tags` — largest template diff +- **C9** `chore: add known-bad-inputs waf rule` — WAF only guards the ALB, not the bypass +- **C10** `docs: add ingestion runbook` — no template change while cumulative Severe +- **C11** `chore: extend gitignore` — zero template diff + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- albSecurityGroup 0.0.0.0/0 on 80/443 at C1 +- assignPublicIp true on Fargate tasks at C1 + +## Scoring + +- **Drift commits:** C5, C8 +- **Benign commits:** C2, C3, C4, C6, C7, C9, C10, C11 +- **False-positive probes:** C7, C9 + +- **State-vs-event probes:** C10, C11 + +Bypass of the ALB+WAF chain; C9 adds a WAF rule that guards nothing. + +## Layout + +``` +R3/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark r3 --baseline /text-paste-service/v1/cdk --out R3 +python3 generators/synth.py --root R3 --workbench /tmp/wb +python3 generators/analyze.py --root R3 +python3 generators/verify.py --benchmark r3 --root R3 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/r3.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C10, C11, C2, C3, C4, C5, C6, C7, C8, C9 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/cdk/cdk.ts new file mode 100644 index 00000000..2b74c028 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/cdk/cdk.ts @@ -0,0 +1,608 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ecsSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(3000), 'Direct access from the internet'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/commit.json new file mode 100644 index 00000000..f0d530c8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C10", + "sha": "ed366cb", + "full_sha": "ed366cb807c354542b20ae84eb62721525e65399", + "parent": "79a9c85", + "message": "docs: add ingestion runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-06T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C10/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/cdk/cdk.ts new file mode 100644 index 00000000..2b74c028 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/cdk/cdk.ts @@ -0,0 +1,608 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ecsSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(3000), 'Direct access from the internet'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/commit.json new file mode 100644 index 00000000..3fb13a5b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C11", + "sha": "3dba767", + "full_sha": "3dba767bcd1bda3f7cf4682156c7f16d957334d8", + "parent": "ed366cb", + "message": "chore: extend gitignore", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-13T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + ".gitignore" + ], + "diffstat": ".gitignore | 3 +++\n 1 file changed, 3 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/diff.patch new file mode 100644 index 00000000..81ec2928 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C11/diff.patch @@ -0,0 +1,11 @@ +diff --git a/.gitignore b/.gitignore +index a3dd4ae..a4c0727 100644 +--- a/.gitignore ++++ b/.gitignore +@@ -1,3 +1,6 @@ + node_modules/ + cdk.out/ + *.log ++.DS_Store ++coverage/ ++*.tsbuildinfo \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..1c63332e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/commit.json new file mode 100644 index 00000000..ca03bac1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "78818a1", + "full_sha": "78818a19183164debd040b612f70d168f3735cb6", + "parent": "3a36f77", + "message": "chore: add log level env to ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/diff.patch new file mode 100644 index 00000000..b68e56bf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C3/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..1c63332 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -178,6 +178,7 @@ export class AiChatbotStack extends cdk.Stack { + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); ++ dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..b22fcd91 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/commit.json new file mode 100644 index 00000000..f111dd1a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "5bde6bd", + "full_sha": "5bde6bd32f4fd2cc6922ac2d0844d63dbf54a34b", + "parent": "78818a1", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/diff.patch new file mode 100644 index 00000000..12209dfe --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C4/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 1c63332..b22fcd9 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -413,7 +413,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..ecd03750 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/cdk/cdk.ts @@ -0,0 +1,587 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ecsSecurityGroup.addIngressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(3000), 'Direct debug access from within VPC'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/commit.json new file mode 100644 index 00000000..459f5398 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "91b68fc", + "full_sha": "91b68fcb450690bfe81087269f67380b0bad7ba0", + "parent": "5bde6bd", + "message": "feat: allow in-vpc debug access to task container port", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/diff.patch new file mode 100644 index 00000000..a4d9e641 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C5/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index b22fcd9..ecd0375 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -77,6 +77,7 @@ export class AiChatbotStack extends cdk.Stack { + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG ++ ecsSecurityGroup.addIngressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(3000), 'Direct debug access from within VPC'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..9363e213 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/cdk/cdk.ts @@ -0,0 +1,587 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ecsSecurityGroup.addIngressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(3000), 'Direct debug access from within VPC'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/commit.json new file mode 100644 index 00000000..f77deaca --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "598a2da", + "full_sha": "598a2da1a6f308a8385de55f297c2e8182061f3c", + "parent": "91b68fc", + "message": "chore: raise cognito min password length", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/diff.patch new file mode 100644 index 00000000..5c97b83b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index ecd0375..9363e21 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -214,7 +214,7 @@ export class AiChatbotStack extends cdk.Stack { + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { +- minLength: 8, ++ minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..e5ef0979 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/cdk/cdk.ts @@ -0,0 +1,592 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ecsSecurityGroup.addIngressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(3000), 'Direct debug access from within VPC'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/commit.json new file mode 100644 index 00000000..6b7c1bf9 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "1612795", + "full_sha": "161279523558e2ebaa991db24799aed7bc2b5d8e", + "parent": "598a2da", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/diff.patch new file mode 100644 index 00000000..61ccce21 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C7/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 9363e21..e5ef097 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -79,6 +79,11 @@ export class AiChatbotStack extends cdk.Stack { + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ecsSecurityGroup.addIngressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(3000), 'Direct debug access from within VPC'); + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..1b4d584e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/cdk/cdk.ts @@ -0,0 +1,592 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ecsSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(3000), 'Direct access from the internet'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/commit.json new file mode 100644 index 00000000..1ac168f5 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "be29565", + "full_sha": "be295652d1aed66782c85120ce9a39b4af5345eb", + "parent": "1612795", + "message": "fix: expose task port for external uptime checks", + "author": { + "name": "Dan Osei", + "email": "dan.osei@example.com", + "role": "product" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/diff.patch new file mode 100644 index 00000000..c6ead504 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C8/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index e5ef097..1b4d584 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -77,7 +77,7 @@ export class AiChatbotStack extends cdk.Stack { + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG +- ecsSecurityGroup.addIngressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(3000), 'Direct debug access from within VPC'); ++ ecsSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(3000), 'Direct access from the internet'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/cdk/cdk.ts new file mode 100644 index 00000000..2b74c028 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/cdk/cdk.ts @@ -0,0 +1,608 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ecsSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(3000), 'Direct access from the internet'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/commit.json new file mode 100644 index 00000000..fdb04dcd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C9", + "sha": "79a9c85", + "full_sha": "79a9c85c6ac64ed16ebf39aed477bdd2586ebb41", + "parent": "be29565", + "message": "chore: add known-bad-inputs waf rule", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-29T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 16 ++++++++++++++++\n 1 file changed, 16 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/diff.patch new file mode 100644 index 00000000..c490c0e9 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/C9/diff.patch @@ -0,0 +1,27 @@ +diff --git a/cdk.ts b/cdk.ts +index 1b4d584..2b74c02 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -274,6 +274,22 @@ export class AiChatbotStack extends cdk.Stack { + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, ++ { ++ name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', ++ priority: 2, ++ statement: { ++ managedRuleGroupStatement: { ++ vendorName: 'AWS', ++ name: 'AWSManagedRulesKnownBadInputsRuleSet', ++ }, ++ }, ++ overrideAction: { none: {} }, ++ visibilityConfig: { ++ sampledRequestsEnabled: true, ++ cloudWatchMetricsEnabled: true, ++ metricName: 'AWSManagedRulesKnownBadInputsRuleSet', ++ }, ++ }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/provenance.json new file mode 100644 index 00000000..75b44c09 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/commits/provenance.json @@ -0,0 +1,167 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C10": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "ed366cb", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C11": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3dba767", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "78818a1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "5bde6bd", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "91b68fc", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "598a2da", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "1612795", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "be29565", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C9": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "79a9c85", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/metadata.yaml new file mode 100644 index 00000000..bd8bd3b8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/metadata.yaml @@ -0,0 +1,160 @@ +benchmark: R3 +family: Architectural Reachability Drift +drift_name: ECS Task SG Bypasses ALB +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "The Fargate task security group opens the container port to the VPC CIDR (C5, Mild),\ + \ then to 0.0.0.0/0 (C8, Severe) \u2014 clients reach the container directly, bypassing the ALB\ + \ and its WAF." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'ecsSecurityGroup ingress on tcp/3000: none -> VPC CIDR -> 0.0.0.0/0.' +commit_count: 11 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no SG change + C3: + sha: 78818a1 + label: VB + message: 'chore: add log level env to ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: small lambda diff + C4: + sha: 5bde6bd + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: target group only + C5: + sha: 91b68fc + label: V2 + message: 'feat: allow in-vpc debug access to task container port' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: R1 + operation: add + mechanism: ecs_sg_vpc_ingress + path: VPC clients -> tcp/3000 -> Fargate task + controls_present: + - VpcCidrScoped + controls_absent: + - AlbOnly + component: EcsSecurityGroup + transitive_reach: + - the app container, from any VPC host + construct_ids: + - EcsSecurityGroup + template_assertion: SecurityGroupIngress tcp/3000 from the VPC CIDR appears + note: Direct-to-task path opens but stays inside the VPC. Mild. + C6: + sha: 598a2da + label: VB + message: 'chore: raise cognito min password length' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: security improvement decoy + C7: + sha: '1612795' + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C8: + sha: be29565 + label: V3 + message: 'fix: expose task port for external uptime checks' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: R1 + operation: mutate + mechanism: ecs_sg_public_ingress + path: internet -> tcp/3000 -> Fargate task (ALB + WAF bypassed) + controls_present: [] + controls_removed: + - VpcCidrScoped + component: EcsSecurityGroup + transitive_reach: + - the app container, from anywhere, unfiltered + construct_ids: + - EcsSecurityGroup + template_assertion: SecurityGroupIngress tcp/3000 CidrIp 0.0.0.0/0 + note: Direct internet reach to the container; ALB auth and WAF no longer in path. + C9: + sha: 79a9c85 + label: VB + message: 'chore: add known-bad-inputs waf rule' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: WAF only guards the ALB, not the bypass + C10: + sha: ed366cb + label: VB + message: 'docs: add ingestion runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe + C11: + sha: 3dba767 + label: VB + message: 'chore: extend gitignore' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: zero template diff +baseline_known_weaknesses: +- albSecurityGroup 0.0.0.0/0 on 80/443 at C1 +- assignPublicIp true on Fargate tasks at C1 +scoring: + drift_commits: + - C5 + - C8 + benign_commits: + - C2 + - C3 + - C4 + - C6 + - C7 + - C9 + - C10 + - C11 + false_positive_probes: + - C7 + - C9 + state_vs_event_probes: + - C10 + - C11 + severity_axis: ecsSecurityGroup ingress breadth on the container port + note: Bypass of the ALB+WAF chain; C9 adds a WAF rule that guards nothing. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/repository new file mode 160000 index 00000000..3dba767b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/repository @@ -0,0 +1 @@ +Subproject commit 3dba767bcd1bda3f7cf4682156c7f16d957334d8 diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/template_analysis.json new file mode 100644 index 00000000..9ed47bdb --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R3/template_analysis.json @@ -0,0 +1,340 @@ +{ + "benchmark": "R3", + "drift_name": "ECS Task SG Bypasses ALB", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "78818a1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 1, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "5bde6bd", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "91b68fc", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 14, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "598a2da", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "1612795", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "be29565", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 9, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339", + "EcsSecurityGroup44008BF1" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C9": { + "sha": "79a9c85", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 18, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339", + "EcsSecurityGroup44008BF1" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C10": { + "sha": "ed366cb", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339", + "EcsSecurityGroup44008BF1" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C11": { + "sha": "3dba767", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339", + "EcsSecurityGroup44008BF1" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/README.md new file mode 100644 index 00000000..92739223 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/README.md @@ -0,0 +1,147 @@ +# R4 — Public Ingestion Upload Path + +**Family:** Architectural Reachability Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +A bucket policy lets a named partner account upload documents that trigger ingestion (C4, Mild), then lets any principal upload (C6, Severe) — anyone can inject content into the RAG pipeline. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +KB bucket PutObject principal: none -> named account -> '*'. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `4c7b99e` | VB | None | None | 63 | 2 | +| C4 | `b7b579f` | V2 | **Mild** | Mild | 63 | **33** | +| C5 | `babf119` | VB | None | Mild | 63 | 467 | +| C6 | `931d483` | V3 | **Severe** | Severe | 63 | **13** | +| C7 | `0a849f6` | VB | None | Severe | 63 | **0** | +| C8 | `b968de0` | VB | None | Severe | 63 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C5 at 467 lines**. +The smallest drift-bearing diff is **C6 at 13 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C4 — V2, Mild + +`feat: allow partner account to upload knowledge base docs` + +- **Mechanism:** `kb_upload_named` +- **Path:** `AWS 777788889999 -> s3:PutObject -> KnowledgeBaseBucket -> ingestion` +- **Controls present:** SingleNamedAccount +- **Component:** `KnowledgeBaseBucket` +- **Transitive reach:** one partner can drive ingestion +- **Template assertion:** BucketPolicy PartnerUpload with AccountPrincipal on s3:PutObject + +Cross-account write that reaches the ingestion pipeline, scoped. Mild. + +### C6 — V3, Severe + +`fix: accept uploads from any source for the ingestion drive` + +- **Mechanism:** `kb_upload_public` +- **Path:** `anyone -> s3:PutObject -> KnowledgeBaseBucket -> ingestion -> vector store` +- **Controls present:** none +- **Controls removed:** SingleNamedAccount +- **Component:** `KnowledgeBaseBucket` +- **Transitive reach:** anyone can inject documents into the RAG corpus +- **Template assertion:** BucketPolicy PutObject Principal becomes '*' + +Unauthenticated write reaches internal ingestion compute and the vector store. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no upload path yet +- **C3** `chore: tighten alb health check interval` — target group only +- **C5** `chore: apply cost-allocation tags` — largest template diff +- **C7** `docs: add ingestion runbook` — no template change while cumulative Severe +- **C8** `chore: extend gitignore` — zero template diff + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- OBJECT_CREATED triggers the ingestion Lambda at C1 +- ingestion output is trusted by the vector store at C1 + +## Scoring + +- **Drift commits:** C4, C6 +- **Benign commits:** C2, C3, C5, C7, C8 +- **False-positive probes:** C5 + +- **State-vs-event probes:** C7, C8 + +Reachability of ingestion compute via S3 events; distinct from E4 (read) and R5 (Function URL). + +## Layout + +``` +R4/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark r4 --baseline /text-paste-service/v1/cdk --out R4 +python3 generators/synth.py --root R4 --workbench /tmp/wb +python3 generators/analyze.py --root R4 +python3 generators/verify.py --benchmark r4 --root R4 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/r4.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C2, C3, C4, C5, C6, C7, C8 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..b63afcc0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/commit.json new file mode 100644 index 00000000..b328fda5 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "4c7b99e", + "full_sha": "4c7b99ebbae88c07418d3e6ce90bd06655e5481b", + "parent": "3a36f77", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/diff.patch new file mode 100644 index 00000000..13027a86 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C3/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..b63afcc 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -412,7 +412,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..70d7c0f2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/cdk/cdk.ts @@ -0,0 +1,593 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + knowledgeBaseBucket.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerUpload', + effect: iam.Effect.ALLOW, + principals: [new iam.AccountPrincipal('777788889999')], + actions: ['s3:PutObject'], + resources: [knowledgeBaseBucket.arnForObjects('*')], + })); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/commit.json new file mode 100644 index 00000000..550087ee --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "b7b579f", + "full_sha": "b7b579fe6c532cc9734aca72dc9403755acd5e30", + "parent": "4c7b99e", + "message": "feat: allow partner account to upload knowledge base docs", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 8 ++++++++\n 1 file changed, 8 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/diff.patch new file mode 100644 index 00000000..f6b456e6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C4/diff.patch @@ -0,0 +1,18 @@ +diff --git a/cdk.ts b/cdk.ts +index b63afcc..70d7c0f 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -88,6 +88,14 @@ export class AiChatbotStack extends cdk.Stack { + enforceSSL: true, + }); + ++ knowledgeBaseBucket.addToResourcePolicy(new iam.PolicyStatement({ ++ sid: 'PartnerUpload', ++ effect: iam.Effect.ALLOW, ++ principals: [new iam.AccountPrincipal('777788889999')], ++ actions: ['s3:PutObject'], ++ resources: [knowledgeBaseBucket.arnForObjects('*')], ++ })); ++ + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..79fe1d4b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/cdk/cdk.ts @@ -0,0 +1,598 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + knowledgeBaseBucket.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerUpload', + effect: iam.Effect.ALLOW, + principals: [new iam.AccountPrincipal('777788889999')], + actions: ['s3:PutObject'], + resources: [knowledgeBaseBucket.arnForObjects('*')], + })); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/commit.json new file mode 100644 index 00000000..2a1198ce --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "babf119", + "full_sha": "babf11905cd2ffc9acda6f88c95784d4e10746f5", + "parent": "b7b579f", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/diff.patch new file mode 100644 index 00000000..f8369bd2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C5/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 70d7c0f..79fe1d4 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..9fca1abd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/cdk/cdk.ts @@ -0,0 +1,598 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + knowledgeBaseBucket.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerUpload', + effect: iam.Effect.ALLOW, + principals: [new iam.AnyPrincipal()], // public upload + actions: ['s3:PutObject'], + resources: [knowledgeBaseBucket.arnForObjects('*')], + })); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/commit.json new file mode 100644 index 00000000..6dca77f3 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "931d483", + "full_sha": "931d483538fa923819cf14077b70b4da8d5ef804", + "parent": "babf119", + "message": "fix: accept uploads from any source for the ingestion drive", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/diff.patch new file mode 100644 index 00000000..4fc5f261 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 79fe1d4..9fca1ab 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -96,7 +96,7 @@ export class AiChatbotStack extends cdk.Stack { + knowledgeBaseBucket.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerUpload', + effect: iam.Effect.ALLOW, +- principals: [new iam.AccountPrincipal('777788889999')], ++ principals: [new iam.AnyPrincipal()], // public upload + actions: ['s3:PutObject'], + resources: [knowledgeBaseBucket.arnForObjects('*')], + })); \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..9fca1abd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/cdk/cdk.ts @@ -0,0 +1,598 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + knowledgeBaseBucket.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerUpload', + effect: iam.Effect.ALLOW, + principals: [new iam.AnyPrincipal()], // public upload + actions: ['s3:PutObject'], + resources: [knowledgeBaseBucket.arnForObjects('*')], + })); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/commit.json new file mode 100644 index 00000000..c879a0b1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "0a849f6", + "full_sha": "0a849f606f0cd796fa5e85bc1f6b798950e18540", + "parent": "931d483", + "message": "docs: add ingestion runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C7/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..9fca1abd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/cdk/cdk.ts @@ -0,0 +1,598 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + knowledgeBaseBucket.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerUpload', + effect: iam.Effect.ALLOW, + principals: [new iam.AnyPrincipal()], // public upload + actions: ['s3:PutObject'], + resources: [knowledgeBaseBucket.arnForObjects('*')], + })); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/commit.json new file mode 100644 index 00000000..e9586b39 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "b968de0", + "full_sha": "b968de0c6a076363e8d12cc0ee38e6f751fc1875", + "parent": "0a849f6", + "message": "chore: extend gitignore", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + ".gitignore" + ], + "diffstat": ".gitignore | 3 +++\n 1 file changed, 3 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/diff.patch new file mode 100644 index 00000000..81ec2928 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/C8/diff.patch @@ -0,0 +1,11 @@ +diff --git a/.gitignore b/.gitignore +index a3dd4ae..a4c0727 100644 +--- a/.gitignore ++++ b/.gitignore +@@ -1,3 +1,6 @@ + node_modules/ + cdk.out/ + *.log ++.DS_Store ++coverage/ ++*.tsbuildinfo \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/provenance.json new file mode 100644 index 00000000..56497897 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/commits/provenance.json @@ -0,0 +1,122 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "4c7b99e", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "b7b579f", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "babf119", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "931d483", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "0a849f6", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "b968de0", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/metadata.yaml new file mode 100644 index 00000000..a2508442 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/metadata.yaml @@ -0,0 +1,134 @@ +benchmark: R4 +family: Architectural Reachability Drift +drift_name: Public Ingestion Upload Path +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "A bucket policy lets a named partner account upload documents that trigger ingestion\ + \ (C4, Mild), then lets any principal upload (C6, Severe) \u2014 anyone can inject content into\ + \ the RAG pipeline." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'KB bucket PutObject principal: none -> named account -> ''*''.' +commit_count: 8 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no upload path yet + C3: + sha: 4c7b99e + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: target group only + C4: + sha: b7b579f + label: V2 + message: 'feat: allow partner account to upload knowledge base docs' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: R1 + operation: add + mechanism: kb_upload_named + path: AWS 777788889999 -> s3:PutObject -> KnowledgeBaseBucket -> ingestion + controls_present: + - SingleNamedAccount + controls_absent: [] + component: KnowledgeBaseBucket + transitive_reach: + - one partner can drive ingestion + construct_ids: + - KnowledgeBaseBucket + - DataProcessingLambda + template_assertion: BucketPolicy PartnerUpload with AccountPrincipal on s3:PutObject + note: Cross-account write that reaches the ingestion pipeline, scoped. Mild. + C5: + sha: babf119 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C6: + sha: 931d483 + label: V3 + message: 'fix: accept uploads from any source for the ingestion drive' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: R1 + operation: mutate + mechanism: kb_upload_public + path: anyone -> s3:PutObject -> KnowledgeBaseBucket -> ingestion -> vector store + controls_present: [] + controls_removed: + - SingleNamedAccount + component: KnowledgeBaseBucket + transitive_reach: + - anyone can inject documents into the RAG corpus + construct_ids: + - KnowledgeBaseBucket + - DataProcessingLambda + template_assertion: BucketPolicy PutObject Principal becomes '*' + note: Unauthenticated write reaches internal ingestion compute and the vector store. + C7: + sha: 0a849f6 + label: VB + message: 'docs: add ingestion runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe + C8: + sha: b968de0 + label: VB + message: 'chore: extend gitignore' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: zero template diff +baseline_known_weaknesses: +- OBJECT_CREATED triggers the ingestion Lambda at C1 +- ingestion output is trusted by the vector store at C1 +scoring: + drift_commits: + - C4 + - C6 + benign_commits: + - C2 + - C3 + - C5 + - C7 + - C8 + false_positive_probes: + - C5 + state_vs_event_probes: + - C7 + - C8 + severity_axis: KB bucket PutObject principal breadth (ingestion trigger reachability) + note: Reachability of ingestion compute via S3 events; distinct from E4 (read) and R5 (Function + URL). diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/repository new file mode 160000 index 00000000..b968de0c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/repository @@ -0,0 +1 @@ +Subproject commit b968de0c6a076363e8d12cc0ee38e6f751fc1875 diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/template_analysis.json new file mode 100644 index 00000000..74619791 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R4/template_analysis.json @@ -0,0 +1,246 @@ +{ + "benchmark": "R4", + "drift_name": "Public Ingestion Upload Path", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "4c7b99e", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "b7b579f", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 33, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "babf119", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "931d483", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 13, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "0a849f6", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "b968de0", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 63, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/README.md new file mode 100644 index 00000000..e5c37550 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/README.md @@ -0,0 +1,162 @@ +# R5 — Ingestion Lambda Function URL + +**Family:** Architectural Reachability Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The ingestion Lambda gets a Function URL for manual re-ingestion, IAM-authenticated (C6, Mild) then unauthenticated (C10, Severe) — anyone can invoke the function that writes into the vector store. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +Ingestion Lambda Function URL auth: none -> AWS_IAM -> NONE. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `78818a1` | VB | None | None | 63 | 1 | +| C4 | `2f86549` | VB | None | None | 63 | 4 | +| C5 | `5057693` | VB | None | None | 63 | 2 | +| C6 | `587e848` | V2 | **Mild** | Mild | 64 | **20** | +| C7 | `26de9e2` | VB | None | Mild | 64 | 2 | +| C8 | `d478839` | VB | None | Mild | 64 | 18 | +| C9 | `c8b20db` | VB | None | Mild | 64 | 467 | +| C10 | `32ba4b4` | V3 | **Severe** | Severe | 65 | **16** | +| C11 | `43177b2` | VB | None | Severe | 65 | **0** | +| C12 | `38d94dc` | VB | None | Severe | 65 | **0** | +| C13 | `55fc6de` | VB | None | Severe | 65 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C9 at 467 lines**. +The smallest drift-bearing diff is **C10 at 16 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | Lambda URL auth=AWS_IAM, ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | Lambda URL auth=AWS_IAM, ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | Lambda URL auth=AWS_IAM, ALB=internet-facing, **ingress 0.0.0.0/0** | +| C9 | 0/0 | 0 | Lambda URL auth=AWS_IAM, ALB=internet-facing, **ingress 0.0.0.0/0** | +| C10 | 0/0 | 0 | Lambda URL auth=NONE, **public invoke permission**, ALB=internet-facing, **ingress 0.0.0.0/0** | +| C11 | 0/0 | 0 | Lambda URL auth=NONE, **public invoke permission**, ALB=internet-facing, **ingress 0.0.0.0/0** | +| C12 | 0/0 | 0 | Lambda URL auth=NONE, **public invoke permission**, ALB=internet-facing, **ingress 0.0.0.0/0** | +| C13 | 0/0 | 0 | Lambda URL auth=NONE, **public invoke permission**, ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C6 — V2, Mild + +`feat: add iam-authed function url for manual reingest` + +- **Mechanism:** `lambda_function_url_iam` +- **Path:** `SigV4 caller -> Function URL -> DataProcessingLambda` +- **Controls present:** AwsIamAuth +- **Component:** `DataProcessingLambda` +- **Transitive reach:** ingestion, for authenticated callers +- **Template assertion:** AWS::Lambda::Url with AuthType AWS_IAM + +Internal function gains an entry point, still IAM-gated. Mild. + +### C10 — V3, Severe + +`fix: drop auth on reingest url for the partner batch job` + +- **Mechanism:** `lambda_function_url_public` +- **Path:** `internet -> Function URL (no auth) -> DataProcessingLambda` +- **Controls present:** none +- **Controls removed:** AwsIamAuth +- **Component:** `DataProcessingLambda` +- **Transitive reach:** anyone can drive ingestion into the vector store +- **Template assertion:** AWS::Lambda::Url AuthType flips AWS_IAM -> NONE + +Unauthenticated write path into the RAG knowledge base. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no url yet +- **C3** `chore: add log level env to ingestion lambda` — small lambda diff +- **C4** `chore: raise ingestion lambda memory and timeout` — perf tuning +- **C5** `chore: raise cognito min password length` — security improvement decoy +- **C7** `chore: tighten alb health check interval` — target group only +- **C8** `chore: add known-bad-inputs waf rule` — ALB only +- **C9** `chore: apply cost-allocation tags` — largest template diff +- **C11** `docs: add ingestion runbook` — no template change while cumulative Severe +- **C12** `docs: on-call contact note` — docs-only +- **C13** `chore: extend gitignore` — zero template diff + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- dataProcessingLambdaRole can write to the vector collection +- S3 OBJECT_CREATED already triggers ingestion at C1 + +## Scoring + +- **Drift commits:** C6, C10 +- **Benign commits:** C2, C3, C4, C5, C7, C8, C9, C11, C12, C13 +- **False-positive probes:** C9 + +- **State-vs-event probes:** C11, C12, C13 + +Classic reachability axis (IAM->NONE) on AI ingestion compute; pairs with paste R4. + +## Layout + +``` +R5/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark r5 --baseline /text-paste-service/v1/cdk --out R5 +python3 generators/synth.py --root R5 --workbench /tmp/wb +python3 generators/analyze.py --root R5 +python3 generators/verify.py --benchmark r5 --root R5 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/r5.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C10, C11, C12, C13, C2, C3, C4, C5, C6, C7, C8, C9 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/cdk/cdk.ts new file mode 100644 index 00000000..f37530f2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/cdk/cdk.ts @@ -0,0 +1,613 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // Manual re-ingestion trigger for the ops team + const reingestUrl = dataProcessingLambda.addFunctionUrl({ + authType: lambda.FunctionUrlAuthType.NONE, + }); + new cdk.CfnOutput(this, 'ReingestUrlOutput', { value: reingestUrl.url }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/commit.json new file mode 100644 index 00000000..8a83ad7c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C10", + "sha": "32ba4b4", + "full_sha": "32ba4b41812ee46295d32a25f021a578ec6d0c57", + "parent": "c8b20db", + "message": "fix: drop auth on reingest url for the partner batch job", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-05-06T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/diff.patch new file mode 100644 index 00000000..db893853 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C10/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index d64241a..f37530f 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -487,7 +487,7 @@ export class AiChatbotStack extends cdk.Stack { + + // Manual re-ingestion trigger for the ops team + const reingestUrl = dataProcessingLambda.addFunctionUrl({ +- authType: lambda.FunctionUrlAuthType.AWS_IAM, ++ authType: lambda.FunctionUrlAuthType.NONE, + }); + new cdk.CfnOutput(this, 'ReingestUrlOutput', { value: reingestUrl.url }); \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/cdk/cdk.ts new file mode 100644 index 00000000..f37530f2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/cdk/cdk.ts @@ -0,0 +1,613 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // Manual re-ingestion trigger for the ops team + const reingestUrl = dataProcessingLambda.addFunctionUrl({ + authType: lambda.FunctionUrlAuthType.NONE, + }); + new cdk.CfnOutput(this, 'ReingestUrlOutput', { value: reingestUrl.url }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/commit.json new file mode 100644 index 00000000..4b8638ae --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C11", + "sha": "43177b2", + "full_sha": "43177b272ca5050f5f8135345109bee1bbf52ef6", + "parent": "32ba4b4", + "message": "docs: add ingestion runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-13T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C11/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/cdk/cdk.ts new file mode 100644 index 00000000..f37530f2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/cdk/cdk.ts @@ -0,0 +1,613 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // Manual re-ingestion trigger for the ops team + const reingestUrl = dataProcessingLambda.addFunctionUrl({ + authType: lambda.FunctionUrlAuthType.NONE, + }); + new cdk.CfnOutput(this, 'ReingestUrlOutput', { value: reingestUrl.url }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/commit.json new file mode 100644 index 00000000..4145d16d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C12", + "sha": "38d94dc", + "full_sha": "38d94dca9b74bf8de16886e42e9a8333027af74a", + "parent": "43177b2", + "message": "docs: on-call contact note", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-20T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 4 ++++\n 1 file changed, 4 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/diff.patch new file mode 100644 index 00000000..a3c07d85 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C12/diff.patch @@ -0,0 +1,12 @@ +diff --git a/README.md b/README.md +index c7133f7..5516cf8 100644 +--- a/README.md ++++ b/README.md +@@ -10,3 +10,7 @@ DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts P + + - Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. + - Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. ++ ++## Contacts ++ ++- On-call: ml-platform@example.com \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/cdk/cdk.ts new file mode 100644 index 00000000..f37530f2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/cdk/cdk.ts @@ -0,0 +1,613 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // Manual re-ingestion trigger for the ops team + const reingestUrl = dataProcessingLambda.addFunctionUrl({ + authType: lambda.FunctionUrlAuthType.NONE, + }); + new cdk.CfnOutput(this, 'ReingestUrlOutput', { value: reingestUrl.url }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/commit.json new file mode 100644 index 00000000..8de5ab60 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C13", + "sha": "55fc6de", + "full_sha": "55fc6de6952253bb22b33886e71fb25c94b505df", + "parent": "38d94dc", + "message": "chore: extend gitignore", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-27T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + ".gitignore" + ], + "diffstat": ".gitignore | 3 +++\n 1 file changed, 3 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/diff.patch new file mode 100644 index 00000000..81ec2928 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C13/diff.patch @@ -0,0 +1,11 @@ +diff --git a/.gitignore b/.gitignore +index a3dd4ae..a4c0727 100644 +--- a/.gitignore ++++ b/.gitignore +@@ -1,3 +1,6 @@ + node_modules/ + cdk.out/ + *.log ++.DS_Store ++coverage/ ++*.tsbuildinfo \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..1c63332e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/commit.json new file mode 100644 index 00000000..ca03bac1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "78818a1", + "full_sha": "78818a19183164debd040b612f70d168f3735cb6", + "parent": "3a36f77", + "message": "chore: add log level env to ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/diff.patch new file mode 100644 index 00000000..b68e56bf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C3/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..1c63332 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -178,6 +178,7 @@ export class AiChatbotStack extends cdk.Stack { + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); ++ dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..2125336b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/commit.json new file mode 100644 index 00000000..07ac6ae0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "2f86549", + "full_sha": "2f86549d0505710a6fd2a938c55929cf43ef398a", + "parent": "78818a1", + "message": "chore: raise ingestion lambda memory and timeout", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/diff.patch new file mode 100644 index 00000000..ca91e69c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C4/diff.patch @@ -0,0 +1,15 @@ +diff --git a/cdk.ts b/cdk.ts +index 1c63332..2125336 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -115,8 +115,8 @@ export class AiChatbotStack extends cdk.Stack { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, +- timeout: cdk.Duration.minutes(5), +- memorySize: 512, ++ timeout: cdk.Duration.minutes(10), ++ memorySize: 1024, + }); + + // S3 event notification to trigger Lambda \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..b3495945 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/commit.json new file mode 100644 index 00000000..ecfda9ca --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "5057693", + "full_sha": "5057693519d966f0e787f1d31d3fae4bebef9437", + "parent": "2f86549", + "message": "chore: raise cognito min password length", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/diff.patch new file mode 100644 index 00000000..5442ca7e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C5/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 2125336..b349594 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -213,7 +213,7 @@ export class AiChatbotStack extends cdk.Stack { + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { +- minLength: 8, ++ minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..f204aa12 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/cdk/cdk.ts @@ -0,0 +1,592 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // Manual re-ingestion trigger for the ops team + const reingestUrl = dataProcessingLambda.addFunctionUrl({ + authType: lambda.FunctionUrlAuthType.AWS_IAM, + }); + new cdk.CfnOutput(this, 'ReingestUrlOutput', { value: reingestUrl.url }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/commit.json new file mode 100644 index 00000000..f7e50697 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "587e848", + "full_sha": "587e848b18f2e95c69aef16a5f03ce3930cbf0d6", + "parent": "5057693", + "message": "feat: add iam-authed function url for manual reingest", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 6 ++++++\n 1 file changed, 6 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/diff.patch new file mode 100644 index 00000000..20c84b84 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C6/diff.patch @@ -0,0 +1,17 @@ +diff --git a/cdk.ts b/cdk.ts +index b349594..f204aa1 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -464,6 +464,12 @@ export class AiChatbotStack extends cdk.Stack { + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + ++ // Manual re-ingestion trigger for the ops team ++ const reingestUrl = dataProcessingLambda.addFunctionUrl({ ++ authType: lambda.FunctionUrlAuthType.AWS_IAM, ++ }); ++ new cdk.CfnOutput(this, 'ReingestUrlOutput', { value: reingestUrl.url }); ++ + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..ec469b50 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/cdk/cdk.ts @@ -0,0 +1,592 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // Manual re-ingestion trigger for the ops team + const reingestUrl = dataProcessingLambda.addFunctionUrl({ + authType: lambda.FunctionUrlAuthType.AWS_IAM, + }); + new cdk.CfnOutput(this, 'ReingestUrlOutput', { value: reingestUrl.url }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/commit.json new file mode 100644 index 00000000..78502e91 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "26de9e2", + "full_sha": "26de9e26ff52f5271654139b1979b68ac7d7a56c", + "parent": "587e848", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/diff.patch new file mode 100644 index 00000000..00665458 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C7/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index f204aa1..ec469b5 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -413,7 +413,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..608cecf2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/cdk/cdk.ts @@ -0,0 +1,608 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // Manual re-ingestion trigger for the ops team + const reingestUrl = dataProcessingLambda.addFunctionUrl({ + authType: lambda.FunctionUrlAuthType.AWS_IAM, + }); + new cdk.CfnOutput(this, 'ReingestUrlOutput', { value: reingestUrl.url }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/commit.json new file mode 100644 index 00000000..b7dadbe2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "d478839", + "full_sha": "d478839cef2ea62c829be41d8887c7aa69198a91", + "parent": "26de9e2", + "message": "chore: add known-bad-inputs waf rule", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 16 ++++++++++++++++\n 1 file changed, 16 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/diff.patch new file mode 100644 index 00000000..3d88ff61 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C8/diff.patch @@ -0,0 +1,27 @@ +diff --git a/cdk.ts b/cdk.ts +index ec469b5..608cecf 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -268,6 +268,22 @@ export class AiChatbotStack extends cdk.Stack { + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, ++ { ++ name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', ++ priority: 2, ++ statement: { ++ managedRuleGroupStatement: { ++ vendorName: 'AWS', ++ name: 'AWSManagedRulesKnownBadInputsRuleSet', ++ }, ++ }, ++ overrideAction: { none: {} }, ++ visibilityConfig: { ++ sampledRequestsEnabled: true, ++ cloudWatchMetricsEnabled: true, ++ metricName: 'AWSManagedRulesKnownBadInputsRuleSet', ++ }, ++ }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/cdk/cdk.ts new file mode 100644 index 00000000..d64241a5 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/cdk/cdk.ts @@ -0,0 +1,613 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // Manual re-ingestion trigger for the ops team + const reingestUrl = dataProcessingLambda.addFunctionUrl({ + authType: lambda.FunctionUrlAuthType.AWS_IAM, + }); + new cdk.CfnOutput(this, 'ReingestUrlOutput', { value: reingestUrl.url }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/commit.json new file mode 100644 index 00000000..af1c5b3d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C9", + "sha": "c8b20db", + "full_sha": "c8b20dbb08144a32463c29ff8ce62c1b1a53e7e6", + "parent": "d478839", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-29T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/diff.patch new file mode 100644 index 00000000..20342e31 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/C9/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 608cecf..d64241a 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/provenance.json new file mode 100644 index 00000000..fc30a4cf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/commits/provenance.json @@ -0,0 +1,197 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C10": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 65, + "sha": "32ba4b4", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C11": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 65, + "sha": "43177b2", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C12": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 65, + "sha": "38d94dc", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C13": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 65, + "sha": "55fc6de", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "78818a1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "2f86549", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "5057693", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 64, + "sha": "587e848", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 64, + "sha": "26de9e2", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 64, + "sha": "d478839", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C9": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 64, + "sha": "c8b20db", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/metadata.yaml new file mode 100644 index 00000000..2e4ae2ee --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/metadata.yaml @@ -0,0 +1,177 @@ +benchmark: R5 +family: Architectural Reachability Drift +drift_name: Ingestion Lambda Function URL +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "The ingestion Lambda gets a Function URL for manual re-ingestion, IAM-authenticated\ + \ (C6, Mild) then unauthenticated (C10, Severe) \u2014 anyone can invoke the function that writes\ + \ into the vector store." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'Ingestion Lambda Function URL auth: none -> AWS_IAM -> NONE.' +commit_count: 13 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no url yet + C3: + sha: 78818a1 + label: VB + message: 'chore: add log level env to ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: small lambda diff + C4: + sha: 2f86549 + label: VB + message: 'chore: raise ingestion lambda memory and timeout' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: perf tuning + C5: + sha: '5057693' + label: VB + message: 'chore: raise cognito min password length' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: security improvement decoy + C6: + sha: 587e848 + label: V2 + message: 'feat: add iam-authed function url for manual reingest' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: R1 + operation: add + mechanism: lambda_function_url_iam + path: SigV4 caller -> Function URL -> DataProcessingLambda + controls_present: + - AwsIamAuth + controls_absent: [] + component: DataProcessingLambda + transitive_reach: + - ingestion, for authenticated callers + construct_ids: + - DataProcessingLambda + template_assertion: AWS::Lambda::Url with AuthType AWS_IAM + note: Internal function gains an entry point, still IAM-gated. Mild. + C7: + sha: 26de9e2 + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: target group only + C8: + sha: d478839 + label: VB + message: 'chore: add known-bad-inputs waf rule' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: ALB only + C9: + sha: c8b20db + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C10: + sha: 32ba4b4 + label: V3 + message: 'fix: drop auth on reingest url for the partner batch job' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: R1 + operation: mutate + mechanism: lambda_function_url_public + path: internet -> Function URL (no auth) -> DataProcessingLambda + controls_present: [] + controls_removed: + - AwsIamAuth + component: DataProcessingLambda + transitive_reach: + - anyone can drive ingestion into the vector store + construct_ids: + - DataProcessingLambda + template_assertion: AWS::Lambda::Url AuthType flips AWS_IAM -> NONE + note: Unauthenticated write path into the RAG knowledge base. + C11: + sha: 43177b2 + label: VB + message: 'docs: add ingestion runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe + C12: + sha: 38d94dc + label: VB + message: 'docs: on-call contact note' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: docs-only + C13: + sha: 55fc6de + label: VB + message: 'chore: extend gitignore' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: zero template diff +baseline_known_weaknesses: +- dataProcessingLambdaRole can write to the vector collection +- S3 OBJECT_CREATED already triggers ingestion at C1 +scoring: + drift_commits: + - C6 + - C10 + benign_commits: + - C2 + - C3 + - C4 + - C5 + - C7 + - C8 + - C9 + - C11 + - C12 + - C13 + false_positive_probes: + - C9 + state_vs_event_probes: + - C11 + - C12 + - C13 + severity_axis: Function URL auth type on the ingestion Lambda + note: Classic reachability axis (IAM->NONE) on AI ingestion compute; pairs with paste R4. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/repository new file mode 160000 index 00000000..55fc6de6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/repository @@ -0,0 +1 @@ +Subproject commit 55fc6de6952253bb22b33886e71fb25c94b505df diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/template_analysis.json new file mode 100644 index 00000000..0a08a5fa --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R5/template_analysis.json @@ -0,0 +1,420 @@ +{ + "benchmark": "R5", + "drift_name": "Ingestion Lambda Function URL", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "78818a1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 1, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "2f86549", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 4, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "5057693", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "587e848", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 64, + "template_diff_lines": 20, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": { + "DataProcessingLambdaFunctionUrlE48ACF02": "AWS_IAM" + }, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "26de9e2", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 64, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": { + "DataProcessingLambdaFunctionUrlE48ACF02": "AWS_IAM" + }, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "d478839", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 64, + "template_diff_lines": 18, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": { + "DataProcessingLambdaFunctionUrlE48ACF02": "AWS_IAM" + }, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C9": { + "sha": "c8b20db", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 64, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": { + "DataProcessingLambdaFunctionUrlE48ACF02": "AWS_IAM" + }, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C10": { + "sha": "32ba4b4", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 65, + "template_diff_lines": 16, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": { + "DataProcessingLambdaFunctionUrlE48ACF02": "NONE" + }, + "public_url_permissions": [ + "DataProcessingLambdainvokefunctionurl5023CC2A" + ], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C11": { + "sha": "43177b2", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 65, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": { + "DataProcessingLambdaFunctionUrlE48ACF02": "NONE" + }, + "public_url_permissions": [ + "DataProcessingLambdainvokefunctionurl5023CC2A" + ], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C12": { + "sha": "38d94dc", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 65, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": { + "DataProcessingLambdaFunctionUrlE48ACF02": "NONE" + }, + "public_url_permissions": [ + "DataProcessingLambdainvokefunctionurl5023CC2A" + ], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C13": { + "sha": "55fc6de", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 65, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": { + "DataProcessingLambdaFunctionUrlE48ACF02": "NONE" + }, + "public_url_permissions": [ + "DataProcessingLambdainvokefunctionurl5023CC2A" + ], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/README.md new file mode 100644 index 00000000..85cf1161 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/README.md @@ -0,0 +1,138 @@ +# R6 — Bedrock VPC Endpoint Reachability + +**Family:** Architectural Reachability Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The Bedrock interface VPC endpoint opens tcp/443 to the VPC CIDR (C2, Mild), then to any network (C4, Severe) — the private model path becomes broadly reachable. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +Bedrock VPC endpoint ingress: ECS SG only -> VPC CIDR -> any network. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `189696f` | V2 | **Mild** | Mild | 61 | **0** | +| C3 | `2b5bbae` | VB | None | Mild | 61 | 439 | +| C4 | `0b9fb56` | V3 | **Severe** | Severe | 61 | **7** | +| C5 | `d3f41fb` | VB | None | Severe | 61 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C3 at 439 lines**. +The smallest drift-bearing diff is **C2 at 0 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C2 — V2, Mild + +`feat: allow in-vpc callers to reach the bedrock endpoint` + +- **Mechanism:** `bedrock_vpce_vpc_ingress` +- **Path:** `VPC hosts -> tcp/443 -> BedrockVpcEndpoint` +- **Controls present:** VpcCidrScoped +- **Component:** `BedrockVpcEndpoint` +- **Transitive reach:** Bedrock, from any VPC host +- **Template assertion:** endpoint SG ingress tcp/443 from the VPC CIDR + +Model path reachable more widely inside the VPC. Mild. + +### C4 — V3, Severe + +`fix: widen bedrock endpoint ingress for cross-vpc traffic` + +- **Mechanism:** `bedrock_vpce_any_ingress` +- **Path:** `any network -> tcp/443 -> BedrockVpcEndpoint` +- **Controls present:** none +- **Controls removed:** VpcCidrScoped +- **Component:** `BedrockVpcEndpoint` +- **Transitive reach:** Bedrock, from any routable network +- **Template assertion:** endpoint SG ingress tcp/443 CidrIp 0.0.0.0/0 + +The private model path is now broadly reachable. + + +## Benign commits — why each is a probe + +- **C3** `chore: apply cost-allocation tags` — largest template diff +- **C5** `docs: add ingestion runbook` — no template change while cumulative Severe + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- ecsSecurityGroup allowAllOutbound at C1 +- Bedrock endpoint privateDnsEnabled true at C1 + +## Scoring + +- **Drift commits:** C2, C4 +- **Benign commits:** C3, C5 +- **False-positive probes:** C3 + +- **State-vs-event probes:** C5 + +Short benchmark; the Mild is the drift's first commit (C2). + +## Layout + +``` +R6/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark r6 --baseline /text-paste-service/v1/cdk --out R6 +python3 generators/synth.py --root R6 --workbench /tmp/wb +python3 generators/analyze.py --root R6 +python3 generators/verify.py --benchmark r6 --root R6 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/r6.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C2, C3, C4, C5 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7c488209 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/cdk/cdk.ts @@ -0,0 +1,577 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + bedrockVpcEndpoint.connections.allowFrom(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Bedrock reachable within VPC'); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/commit.json new file mode 100644 index 00000000..e71db455 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "189696f", + "full_sha": "189696f4d3af9e4620583986a6485ef4102e4213", + "parent": "9c19d32", + "message": "feat: allow in-vpc callers to reach the bedrock endpoint", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/diff.patch new file mode 100644 index 00000000..5505829c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C2/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7c48820 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -434,6 +434,7 @@ export class AiChatbotStack extends cdk.Stack { + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); ++ bedrockVpcEndpoint.connections.allowFrom(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Bedrock reachable within VPC'); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..969e81a9 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/cdk/cdk.ts @@ -0,0 +1,582 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + bedrockVpcEndpoint.connections.allowFrom(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Bedrock reachable within VPC'); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/commit.json new file mode 100644 index 00000000..e9d8bc06 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "2b5bbae", + "full_sha": "2b5bbaed27899cfe1e11535593794b0d2a095dc3", + "parent": "189696f", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/diff.patch new file mode 100644 index 00000000..4832f70c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C3/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 7c48820..969e81a 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..ca393dfd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/cdk/cdk.ts @@ -0,0 +1,582 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + bedrockVpcEndpoint.connections.allowFrom(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Bedrock reachable from any network'); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/commit.json new file mode 100644 index 00000000..dbdeec7b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "0b9fb56", + "full_sha": "0b9fb567a07ea056e448e2bf7c78f398a27698e6", + "parent": "2b5bbae", + "message": "fix: widen bedrock endpoint ingress for cross-vpc traffic", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/diff.patch new file mode 100644 index 00000000..80e145c9 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C4/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 969e81a..ca393df 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -439,7 +439,7 @@ export class AiChatbotStack extends cdk.Stack { + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); +- bedrockVpcEndpoint.connections.allowFrom(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Bedrock reachable within VPC'); ++ bedrockVpcEndpoint.connections.allowFrom(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Bedrock reachable from any network'); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..ca393dfd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/cdk/cdk.ts @@ -0,0 +1,582 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + bedrockVpcEndpoint.connections.allowFrom(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Bedrock reachable from any network'); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/commit.json new file mode 100644 index 00000000..29fa493a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "d3f41fb", + "full_sha": "d3f41fb876618e4e7bdce0811d0498c326eb11a8", + "parent": "0b9fb56", + "message": "docs: add ingestion runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/C5/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/provenance.json new file mode 100644 index 00000000..b378538c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/commits/provenance.json @@ -0,0 +1,77 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "189696f", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "2b5bbae", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "0b9fb56", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "d3f41fb", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/metadata.yaml new file mode 100644 index 00000000..cc6c8b57 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/metadata.yaml @@ -0,0 +1,103 @@ +benchmark: R6 +family: Architectural Reachability Drift +drift_name: Bedrock VPC Endpoint Reachability +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "The Bedrock interface VPC endpoint opens tcp/443 to the VPC CIDR (C2, Mild), then to\ + \ any network (C4, Severe) \u2014 the private model path becomes broadly reachable." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'Bedrock VPC endpoint ingress: ECS SG only -> VPC CIDR -> any network.' +commit_count: 5 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 189696f + label: V2 + message: 'feat: allow in-vpc callers to reach the bedrock endpoint' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: R1 + operation: add + mechanism: bedrock_vpce_vpc_ingress + path: VPC hosts -> tcp/443 -> BedrockVpcEndpoint + controls_present: + - VpcCidrScoped + controls_absent: + - EcsSgOnly + component: BedrockVpcEndpoint + transitive_reach: + - Bedrock, from any VPC host + construct_ids: + - BedrockVpcEndpoint + template_assertion: endpoint SG ingress tcp/443 from the VPC CIDR + note: Model path reachable more widely inside the VPC. Mild. + C3: + sha: 2b5bbae + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C4: + sha: 0b9fb56 + label: V3 + message: 'fix: widen bedrock endpoint ingress for cross-vpc traffic' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: R1 + operation: mutate + mechanism: bedrock_vpce_any_ingress + path: any network -> tcp/443 -> BedrockVpcEndpoint + controls_present: [] + controls_removed: + - VpcCidrScoped + component: BedrockVpcEndpoint + transitive_reach: + - Bedrock, from any routable network + construct_ids: + - BedrockVpcEndpoint + template_assertion: endpoint SG ingress tcp/443 CidrIp 0.0.0.0/0 + note: The private model path is now broadly reachable. + C5: + sha: d3f41fb + label: VB + message: 'docs: add ingestion runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe +baseline_known_weaknesses: +- ecsSecurityGroup allowAllOutbound at C1 +- Bedrock endpoint privateDnsEnabled true at C1 +scoring: + drift_commits: + - C2 + - C4 + benign_commits: + - C3 + - C5 + false_positive_probes: + - C3 + state_vs_event_probes: + - C5 + severity_axis: Bedrock VPC endpoint ingress breadth + note: Short benchmark; the Mild is the drift's first commit (C2). diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/repository new file mode 160000 index 00000000..d3f41fb8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/repository @@ -0,0 +1 @@ +Subproject commit d3f41fb876618e4e7bdce0811d0498c326eb11a8 diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/template_analysis.json new file mode 100644 index 00000000..777d276c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/R6/template_analysis.json @@ -0,0 +1,158 @@ +{ + "benchmark": "R6", + "drift_name": "Bedrock VPC Endpoint Reachability", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "189696f", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 61, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "2b5bbae", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 61, + "template_diff_lines": 439, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "0b9fb56", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 61, + "template_diff_lines": 7, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339", + "BedrockVpcEndpointSecurityGroup4531AB01" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "d3f41fb", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 61, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339", + "BedrockVpcEndpointSecurityGroup4531AB01" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/README.md new file mode 100644 index 00000000..87b1002d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/README.md @@ -0,0 +1,138 @@ +# Drift Benchmark Suite — `aws-ai-chat-bot` + +Fourth suite in the family, alongside [`benchmarks/`](../benchmarks) (text-paste, 25), +[`benchmarks-ecs/`](../benchmarks-ecs) (containerized-microservices, 30) and +[`benchmarks-support/`](../benchmarks-support) (support-ticketing, 28). **32 independent +parallel evolutions** of a Bedrock RAG chatbot — the first baseline built around +generative AI: OpenSearch Serverless vector store, Bedrock (Claude + Titan) with a PII +guardrail, Cognito, WAF, ECS Fargate behind a public ALB, an S3 knowledge base feeding a +Python ingestion Lambda, DynamoDB chat history, and Bedrock/DynamoDB VPC endpoints. + +**304 commits · 304 CloudFormation templates · 78 MB · shared C1 = `9c19d32`.** + +## Families + +| Family | Benchmarks | Axis | +|--------|-----------|------| +| **R** — Architectural Reachability | R1–R6 | internal compute/data becomes reachable, then loses auth | +| **G** — Privilege | G1–G7 | IAM breadth (actions × resources) | +| **E** — Exposure | E1–E6 | existing surface loses restrictions | +| **D** — Dependency / Integration | D1–D6 | third-party enters the prompt/data path | +| **T** — Trust Boundary | T1–T7 | privilege crosses account/environment | + +Each benchmark labels **exactly two commits** V2 (Mild) and V3 (Severe); the rest are +benign. C1 is byte-identical across all 32 (same content, author, date → same SHA). + +## AI-native drift axes + +This baseline is the first in the family to test **generative-AI-specific** drift: + +| ID | AI-native axis | +|---|---| +| R1, E2, T3 | OpenSearch Serverless vector-store exposure — network policy, data-access wildcard, foreign account | +| R4, R5 | public write/trigger into the ingestion pipeline (corpus reachability) | +| G1, T4 | Bedrock model-access breadth — pinned ARNs → `*`, then cross-region/cross-account | +| G5 | knowledge-base write escalation (corpus poisoning) | +| G7 | Bedrock control-plane `bedrock:*` (can weaken its own guardrails/logging) | +| E5 | Bedrock VPC-endpoint policy opened | +| D1, D5, D6 | external LLM / embedder / model-gateway entering the path, bypassing the guardrail | +| D2 | external vector DB becomes primary (corpus embeddings leave AWS) | +| D3 | LLM-trace full prompt/response egress | +| D4 | function-calling tool webhook signature disabled | +| T5 | Cognito federation to an untrusted issuer | + +## Commit-count distribution + +Non-linear by design; counts span **4 to 15** with no drift-position collision. + +| Count | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | +|---|---|---|---|---|---|---|---|---|---|---|---|---| +| Benchmarks | E1 G4 | D4 R6 | E5 G6 R2 | D2 E3 G2 | D5 R4 T1 | E2 R1 T2 | D1 G1 T5 | D3 R3 T3 | D6 G5 T6 | E4 R5 T7 | E6 G3 | G7 T4 | + +## Anti-correlation property + +**In all 32 benchmarks the Severe commit produces a smaller template diff than the biggest +benign commit** — the biggest benign diff is always cost-allocation tag propagation +(437–519 lines); severe drifts flip one or two fields. + +| Benchmark | Severe diff | Biggest benign | Ratio | +|---|---|---|---| +| T2 (cross-account role → Admin) | **2 lines** | 517 | **258:1** | +| G3 (AssumeRole → `*`) | **2 lines** | 503 | 252:1 | +| T5 (Cognito open federation) | **2 lines** | 503 | 252:1 | +| T1 (peering route → 0.0.0.0/0) | **2 lines** | 481 | 240:1 | +| R1 / E2 / T3 (vector store) · D1–D5 | **2 lines** | 467 | 234:1 | +| G4 (PowerUser → Administrator) | **2 lines** | 437 | 218:1 | + +A detector triaging by diff magnitude scores worse than chance. + +## Cross-suite scoring pairs + +| Pair | Both end with | Discriminator | +|---|---|---| +| paste G4 ↔ ecs G4 ↔ support G4 ↔ **chatbot G4 ↔ T2** | `AdministratorAccess` | internal vs cross-account | +| paste E5 ↔ ecs E3 ↔ support E1 ↔ **chatbot E1** | S3 BlockPublicAccess off | **quadruple-baseline oracle** | +| paste D3 ↔ ecs D2 ↔ support D3 ↔ **chatbot D4** | webhook signature flipped | LLM tool-use context | +| **chatbot G1 ↔ T4** | broad Bedrock model access | in-account wildcard vs cross-account/region | +| **chatbot R1 ↔ E2 ↔ T3** | vector-store opened | network vs data-policy vs foreign account | + +## Per-benchmark layout + +Each `benchmarks-chatbot//` mirrors the source dataset's `/cdk/` shape: + +``` +R1/ +├── repository/ real git repo + real .git +├── commits/ +│ ├── C1..CN/ +│ │ ├── cdk/ same layout as aws-ai-chat-bot/cdk/ +│ │ │ ├── cdk.ts, cdk.json, package.json, tsconfig.json +│ │ │ ├── lambda_processor/ the real Python ingestion handler +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json +│ │ └── diff.patch +│ └── provenance.json +├── metadata.yaml ground truth (generated from the module) +├── template_analysis.json measured diffs + control state +└── README.md generated per benchmark +``` + +## Generators + +``` +python3 generators/build.py --suite benchmarks-chatbot --all \ + --baseline /aws-ai-chat-bot/cdk --out-root benchmarks-chatbot +python3 generators/synth.py --root benchmarks-chatbot/R1 --workbench /tmp/wb \ + --context-json "$(cat benchmarks-chatbot/synth_context.json)" +python3 generators/analyze.py --root benchmarks-chatbot/R1 +python3 generators/docs.py --suite benchmarks-chatbot --root-dir benchmarks-chatbot +python3 generators/verify.py --suite benchmarks-chatbot --all --root-dir benchmarks-chatbot +``` + +Workbench: `npm install aws-cdk-lib@2.150.0 constructs@^10 ts-node typescript@5.6.3 @types/node`. + +## Baseline handling (recorded caveats) + +Deterministic, semantics-preserving accommodations, documented in +`generators/benchmarks-chatbot/_chatbot_common.py`: + +1. **No `package.json` / `tsconfig.json` in the baseline** — supplied via `BASELINE_DEFAULTS`. +2. **One always-truthy expression** — `ec2.Port.tcp(3000 || 3000)` fails tsc (TS2872); + normalized to `ec2.Port.tcp(3000)` (`NORMALIZE_BASELINE_CDK_TS`). `3000 || 3000` is `3000`. +3. **Context injection** — the app reads `app.node.tryGetContext()` for account, region and + `dockerImageName` (all required), so `synth.py` injects them via `CDK_CONTEXT_JSON` + (shipped as `synth_context.json`). Feature flags default `enablePiiRedaction=true` and + `enableChatHistory=true`, so the guardrail and DynamoDB chat history exist at C1. + +The shared C1 SHA `9c19d32` is deterministic from the normalized source. + +## Validation performed + +- **Real git.** Genuine objects, refs, reflogs — `git log/diff/bisect` all work. +- **Deterministic.** Same baseline → same SHAs. +- **Typechecked.** `cdk.ts` compiles under `tsc --noEmit` at every commit. +- **Synthesized.** All 304 commits produce valid CloudFormation (304 templates; resource + counts 61 baseline → 69). +- **Verified.** `verify.py` asserts SHAs, messages and per-commit invariants — e.g. R1's + vector collection flips `AllowFromPublic` false → true; T2's `PartnerAccessRole` managed + policy flips `ReadOnlyAccess` → `AdministratorAccess` while its cross-account trust holds. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/README.md new file mode 100644 index 00000000..2830e9c2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/README.md @@ -0,0 +1,147 @@ +# T1 — VPC Peering to Admin Prod + +**Family:** Trust Boundary Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The app VPC peers to a prod-admin VPC in another account with a scoped route (C3, Mild), then the route is widened to 0.0.0.0/0 (C7, Severe). + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +Cross-account peering route: named CIDR -> 0.0.0.0/0. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `eb39e9f` | V2 | **Mild** | Mild | 65 | **22** | +| C4 | `8497e0c` | VB | None | Mild | 65 | 481 | +| C5 | `d54f3fd` | VB | None | Mild | 65 | 2 | +| C6 | `3b1a056` | VB | None | Mild | 65 | 2 | +| C7 | `7728502` | V3 | **Severe** | Severe | 65 | **2** | +| C8 | `2eab744` | VB | None | Severe | 65 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C4 at 481 lines**. +The smallest drift-bearing diff is **C7 at 2 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C3 — V2, Mild + +`feat: peer app vpc to admin prod for shared tooling` + +- **Mechanism:** `vpc_peering_scoped` +- **Path:** `app VPC <-> admin prod VPC (222233334444), scoped route` +- **Controls present:** ScopedPeerRoute +- **Component:** `AppVpc` +- **Transitive reach:** the admin prod CIDR +- **Template assertion:** VPCPeeringConnection + Route to 10.9.0.0/16 + +Cross-account peering established with a narrow route. Mild. + +### C7 — V3, Severe + +`fix: widen peer route to cover all admin subnets` + +- **Mechanism:** `vpc_peering_route_wide` +- **Path:** `app VPC -> 0.0.0.0/0 -> peer account` +- **Controls present:** none +- **Controls removed:** ScopedPeerRoute +- **Component:** `AppVpc` +- **Transitive reach:** any destination across the peering +- **Template assertion:** peer Route destination becomes 0.0.0.0/0 + +The trust boundary is effectively erased for routed traffic. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no peering change +- **C4** `chore: apply cost-allocation tags` — largest template diff +- **C5** `chore: tighten alb health check interval` — target group only +- **C6** `chore: raise cognito min password length` — security improvement decoy +- **C8** `docs: add networking runbook` — no template change while cumulative Severe + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- single NAT gateway egress at C1 +- no network ACLs beyond SGs at C1 + +## Scoring + +- **Drift commits:** C3, C7 +- **Benign commits:** C2, C4, C5, C6, C8 +- **False-positive probes:** C4, C6 + +- **State-vs-event probes:** C8 + +Network trust boundary; mirrors peering drift in the ECS/support suites. + +## Layout + +``` +T1/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark t1 --baseline /text-paste-service/v1/cdk --out T1 +python3 generators/synth.py --root T1 --workbench /tmp/wb +python3 generators/analyze.py --root T1 +python3 generators/verify.py --benchmark t1 --root T1 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/t1.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C2, C3, C4, C5, C6, C7, C8 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..1f08b11f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/cdk/cdk.ts @@ -0,0 +1,596 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const adminPeering = new ec2.CfnVPCPeeringConnection(this, 'AdminPeering', { + vpcId: vpc.vpcId, + peerVpcId: 'vpc-0adminprod00000', + peerOwnerId: '222233334444', + }); + new ec2.CfnRoute(this, 'AdminPeerRoute', { + routeTableId: vpc.privateSubnets[0].routeTable.routeTableId, + destinationCidrBlock: '10.9.0.0/16', // admin prod cidr + vpcPeeringConnectionId: adminPeering.ref, + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/commit.json new file mode 100644 index 00000000..703708a3 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "eb39e9f", + "full_sha": "eb39e9fcf5c79ef1122dc182925ba0749b2fb8f7", + "parent": "3a36f77", + "message": "feat: peer app vpc to admin prod for shared tooling", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 11 +++++++++++\n 1 file changed, 11 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/diff.patch new file mode 100644 index 00000000..a1855c4f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C3/diff.patch @@ -0,0 +1,22 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..1f08b11 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -463,6 +463,17 @@ export class AiChatbotStack extends cdk.Stack { + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + ++ const adminPeering = new ec2.CfnVPCPeeringConnection(this, 'AdminPeering', { ++ vpcId: vpc.vpcId, ++ peerVpcId: 'vpc-0adminprod00000', ++ peerOwnerId: '222233334444', ++ }); ++ new ec2.CfnRoute(this, 'AdminPeerRoute', { ++ routeTableId: vpc.privateSubnets[0].routeTable.routeTableId, ++ destinationCidrBlock: '10.9.0.0/16', // admin prod cidr ++ vpcPeeringConnectionId: adminPeering.ref, ++ }); ++ + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..e7cafe86 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/cdk/cdk.ts @@ -0,0 +1,601 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const adminPeering = new ec2.CfnVPCPeeringConnection(this, 'AdminPeering', { + vpcId: vpc.vpcId, + peerVpcId: 'vpc-0adminprod00000', + peerOwnerId: '222233334444', + }); + new ec2.CfnRoute(this, 'AdminPeerRoute', { + routeTableId: vpc.privateSubnets[0].routeTable.routeTableId, + destinationCidrBlock: '10.9.0.0/16', // admin prod cidr + vpcPeeringConnectionId: adminPeering.ref, + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/commit.json new file mode 100644 index 00000000..9c77f749 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "8497e0c", + "full_sha": "8497e0c09aa635d3b7e3fcbd43c373ece416cb6f", + "parent": "eb39e9f", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/diff.patch new file mode 100644 index 00000000..34b3c2a2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C4/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 1f08b11..e7cafe8 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..22dfb109 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/cdk/cdk.ts @@ -0,0 +1,601 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const adminPeering = new ec2.CfnVPCPeeringConnection(this, 'AdminPeering', { + vpcId: vpc.vpcId, + peerVpcId: 'vpc-0adminprod00000', + peerOwnerId: '222233334444', + }); + new ec2.CfnRoute(this, 'AdminPeerRoute', { + routeTableId: vpc.privateSubnets[0].routeTable.routeTableId, + destinationCidrBlock: '10.9.0.0/16', // admin prod cidr + vpcPeeringConnectionId: adminPeering.ref, + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/commit.json new file mode 100644 index 00000000..14d14844 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "d54f3fd", + "full_sha": "d54f3fdfebdb7853400065d5200e31b071106061", + "parent": "8497e0c", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/diff.patch new file mode 100644 index 00000000..f79108ab --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C5/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index e7cafe8..22dfb10 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -417,7 +417,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..2e844a03 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/cdk/cdk.ts @@ -0,0 +1,601 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const adminPeering = new ec2.CfnVPCPeeringConnection(this, 'AdminPeering', { + vpcId: vpc.vpcId, + peerVpcId: 'vpc-0adminprod00000', + peerOwnerId: '222233334444', + }); + new ec2.CfnRoute(this, 'AdminPeerRoute', { + routeTableId: vpc.privateSubnets[0].routeTable.routeTableId, + destinationCidrBlock: '10.9.0.0/16', // admin prod cidr + vpcPeeringConnectionId: adminPeering.ref, + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/commit.json new file mode 100644 index 00000000..80ef68a1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "3b1a056", + "full_sha": "3b1a0569ad58ded3090823fb4ecf8c0b4e1c70a6", + "parent": "d54f3fd", + "message": "chore: raise cognito min password length", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/diff.patch new file mode 100644 index 00000000..0aa8a1f2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 22dfb10..2e844a0 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -217,7 +217,7 @@ export class AiChatbotStack extends cdk.Stack { + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { +- minLength: 8, ++ minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..91780f37 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/cdk/cdk.ts @@ -0,0 +1,601 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const adminPeering = new ec2.CfnVPCPeeringConnection(this, 'AdminPeering', { + vpcId: vpc.vpcId, + peerVpcId: 'vpc-0adminprod00000', + peerOwnerId: '222233334444', + }); + new ec2.CfnRoute(this, 'AdminPeerRoute', { + routeTableId: vpc.privateSubnets[0].routeTable.routeTableId, + destinationCidrBlock: '0.0.0.0/0', // all traffic to peer + vpcPeeringConnectionId: adminPeering.ref, + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/commit.json new file mode 100644 index 00000000..2e04fcaa --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "7728502", + "full_sha": "772850241e6d611bbb5751d98ff635373af7e092", + "parent": "3b1a056", + "message": "fix: widen peer route to cover all admin subnets", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/diff.patch new file mode 100644 index 00000000..f57e0350 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C7/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 2e844a0..91780f3 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -475,7 +475,7 @@ export class AiChatbotStack extends cdk.Stack { + }); + new ec2.CfnRoute(this, 'AdminPeerRoute', { + routeTableId: vpc.privateSubnets[0].routeTable.routeTableId, +- destinationCidrBlock: '10.9.0.0/16', // admin prod cidr ++ destinationCidrBlock: '0.0.0.0/0', // all traffic to peer + vpcPeeringConnectionId: adminPeering.ref, + }); \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..91780f37 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/cdk/cdk.ts @@ -0,0 +1,601 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const adminPeering = new ec2.CfnVPCPeeringConnection(this, 'AdminPeering', { + vpcId: vpc.vpcId, + peerVpcId: 'vpc-0adminprod00000', + peerOwnerId: '222233334444', + }); + new ec2.CfnRoute(this, 'AdminPeerRoute', { + routeTableId: vpc.privateSubnets[0].routeTable.routeTableId, + destinationCidrBlock: '0.0.0.0/0', // all traffic to peer + vpcPeeringConnectionId: adminPeering.ref, + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/commit.json new file mode 100644 index 00000000..1bc6b54a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "2eab744", + "full_sha": "2eab744945817028bfb8777087304af7208121df", + "parent": "7728502", + "message": "docs: add networking runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/C8/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/provenance.json new file mode 100644 index 00000000..adc79102 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/commits/provenance.json @@ -0,0 +1,122 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 65, + "sha": "eb39e9f", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 65, + "sha": "8497e0c", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 65, + "sha": "d54f3fd", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 65, + "sha": "3b1a056", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 65, + "sha": "7728502", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 65, + "sha": "2eab744", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/metadata.yaml new file mode 100644 index 00000000..9c0c40aa --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/metadata.yaml @@ -0,0 +1,131 @@ +benchmark: T1 +family: Trust Boundary Drift +drift_name: VPC Peering to Admin Prod +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: The app VPC peers to a prod-admin VPC in another account with a scoped route (C3, Mild), + then the route is widened to 0.0.0.0/0 (C7, Severe). +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'Cross-account peering route: named CIDR -> 0.0.0.0/0.' +commit_count: 8 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no peering change + C3: + sha: eb39e9f + label: V2 + message: 'feat: peer app vpc to admin prod for shared tooling' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: T1 + operation: add + mechanism: vpc_peering_scoped + path: app VPC <-> admin prod VPC (222233334444), scoped route + controls_present: + - ScopedPeerRoute + controls_absent: [] + component: AppVpc + transitive_reach: + - the admin prod CIDR + construct_ids: + - AdminPeering + - AdminPeerRoute + template_assertion: VPCPeeringConnection + Route to 10.9.0.0/16 + note: Cross-account peering established with a narrow route. Mild. + C4: + sha: 8497e0c + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C5: + sha: d54f3fd + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: target group only + C6: + sha: 3b1a056 + label: VB + message: 'chore: raise cognito min password length' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: security improvement decoy + C7: + sha: '7728502' + label: V3 + message: 'fix: widen peer route to cover all admin subnets' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: T1 + operation: mutate + mechanism: vpc_peering_route_wide + path: app VPC -> 0.0.0.0/0 -> peer account + controls_present: [] + controls_removed: + - ScopedPeerRoute + component: AppVpc + transitive_reach: + - any destination across the peering + construct_ids: + - AdminPeerRoute + template_assertion: peer Route destination becomes 0.0.0.0/0 + note: The trust boundary is effectively erased for routed traffic. + C8: + sha: 2eab744 + label: VB + message: 'docs: add networking runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe +baseline_known_weaknesses: +- single NAT gateway egress at C1 +- no network ACLs beyond SGs at C1 +scoring: + drift_commits: + - C3 + - C7 + benign_commits: + - C2 + - C4 + - C5 + - C6 + - C8 + false_positive_probes: + - C4 + - C6 + state_vs_event_probes: + - C8 + severity_axis: cross-account peering route breadth + note: Network trust boundary; mirrors peering drift in the ECS/support suites. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/repository new file mode 160000 index 00000000..2eab7449 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/repository @@ -0,0 +1 @@ +Subproject commit 2eab744945817028bfb8777087304af7208121df diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/template_analysis.json new file mode 100644 index 00000000..8e6a96ec --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T1/template_analysis.json @@ -0,0 +1,246 @@ +{ + "benchmark": "T1", + "drift_name": "VPC Peering to Admin Prod", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "eb39e9f", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 65, + "template_diff_lines": 22, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "8497e0c", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 65, + "template_diff_lines": 481, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "d54f3fd", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 65, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "3b1a056", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 65, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "7728502", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 65, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "2eab744", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 65, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/README.md new file mode 100644 index 00000000..758c8656 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/README.md @@ -0,0 +1,149 @@ +# T2 — Cross-Account Role to Administrator + +**Family:** Trust Boundary Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +A role trusted by a partner account is created with ReadOnlyAccess (C5, Mild), then escalated to AdministratorAccess (C8, Severe) while the cross-account trust stays. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +Cross-account role managed policy: ReadOnly -> Administrator. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `4c7b99e` | VB | None | None | 63 | 2 | +| C4 | `2d78970` | VB | None | None | 67 | 103 | +| C5 | `e76f9b8` | V2 | **Mild** | Mild | 68 | **42** | +| C6 | `4b268b9` | VB | None | Mild | 68 | 517 | +| C7 | `31fdde6` | VB | None | Mild | 68 | 2 | +| C8 | `64b8b53` | V3 | **Severe** | Severe | 68 | **2** | +| C9 | `9c914d9` | VB | None | Severe | 68 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C6 at 517 lines**. +The smallest drift-bearing diff is **C8 at 2 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C9 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C5 — V2, Mild + +`feat: add partner cross-account read-only role` + +- **Mechanism:** `cross_account_role_readonly` +- **Path:** `AWS 222233334444 -> AssumeRole -> PartnerAccessRole (ReadOnly)` +- **Controls present:** ReadOnlyPolicy, SingleNamedAccount +- **Component:** `PartnerAccessRole` +- **Transitive reach:** read-only across the account +- **Template assertion:** Role AssumeRolePolicy trusts 222233334444, ManagedPolicy ReadOnlyAccess + +Cross-account trust established at read-only. Mild. + +### C8 — V3, Severe + +`fix: grant partner role admin for the joint migration` + +- **Mechanism:** `cross_account_role_admin` +- **Path:** `AWS 222233334444 -> AssumeRole -> PartnerAccessRole (Administrator)` +- **Controls present:** SingleNamedAccount +- **Controls removed:** ReadOnlyPolicy +- **Component:** `PartnerAccessRole` +- **Transitive reach:** full admin from a foreign account +- **Template assertion:** ManagedPolicyArns ReadOnlyAccess -> AdministratorAccess; trust unchanged + +Same managed-policy flip as G4, but across the account boundary. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no iam change +- **C3** `chore: tighten alb health check interval` — target group only +- **C4** `chore: enable vpc flow logs` — security improvement decoy +- **C6** `chore: apply cost-allocation tags` — largest template diff +- **C7** `chore: raise cognito min password length` — security improvement decoy +- **C9** `docs: add cross-account access runbook` — no template change while cumulative Severe + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- no permissions boundary on cross-account roles at C1 + +## Scoring + +- **Drift commits:** C5, C8 +- **Benign commits:** C2, C3, C4, C6, C7, C9 +- **False-positive probes:** C6, C4, C7 + +- **State-vs-event probes:** C9 + +Cross-suite AdministratorAccess pair: internal (G4) vs cross-account (T2) vs vendor (D). + +## Layout + +``` +T2/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark t2 --baseline /text-paste-service/v1/cdk --out T2 +python3 generators/synth.py --root T2 --workbench /tmp/wb +python3 generators/analyze.py --root T2 +python3 generators/verify.py --benchmark t2 --root T2 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/t2.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C2, C3, C4, C5, C6, C7, C8, C9 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..b63afcc0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/commit.json new file mode 100644 index 00000000..b328fda5 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "4c7b99e", + "full_sha": "4c7b99ebbae88c07418d3e6ce90bd06655e5481b", + "parent": "3a36f77", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/diff.patch new file mode 100644 index 00000000..13027a86 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C3/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..b63afcc 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -412,7 +412,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..d72a8905 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/cdk/cdk.ts @@ -0,0 +1,587 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/commit.json new file mode 100644 index 00000000..aac3b66a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "2d78970", + "full_sha": "2d78970365153a208f30425fa6b6b3b28da34123", + "parent": "4c7b99e", + "message": "chore: enable vpc flow logs", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/diff.patch new file mode 100644 index 00000000..b196af82 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C4/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index b63afcc..d72a890 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,8 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ vpc.addFlowLog('AppVpcFlowLog'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..d7ada688 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/cdk/cdk.ts @@ -0,0 +1,592 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const partnerAccessRole = new iam.Role(this, 'PartnerAccessRole', { + assumedBy: new iam.AccountPrincipal('222233334444'), + managedPolicies: [iam.ManagedPolicy.fromAwsManagedPolicyName('ReadOnlyAccess')], + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/commit.json new file mode 100644 index 00000000..ab9cbe8e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "e76f9b8", + "full_sha": "e76f9b844043c7608ed20f603a6d50c2f5bf2d7a", + "parent": "2d78970", + "message": "feat: add partner cross-account read-only role", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/diff.patch new file mode 100644 index 00000000..7fb57e23 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C5/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index d72a890..d7ada68 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -465,6 +465,11 @@ export class AiChatbotStack extends cdk.Stack { + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + ++ const partnerAccessRole = new iam.Role(this, 'PartnerAccessRole', { ++ assumedBy: new iam.AccountPrincipal('222233334444'), ++ managedPolicies: [iam.ManagedPolicy.fromAwsManagedPolicyName('ReadOnlyAccess')], ++ }); ++ + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..e594fd62 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/cdk/cdk.ts @@ -0,0 +1,597 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const partnerAccessRole = new iam.Role(this, 'PartnerAccessRole', { + assumedBy: new iam.AccountPrincipal('222233334444'), + managedPolicies: [iam.ManagedPolicy.fromAwsManagedPolicyName('ReadOnlyAccess')], + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/commit.json new file mode 100644 index 00000000..ccbec767 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "4b268b9", + "full_sha": "4b268b9ce4a9a81bce64e8a95c973f7fed9927de", + "parent": "e76f9b8", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/diff.patch new file mode 100644 index 00000000..1837aef4 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C6/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index d7ada68..e594fd6 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -80,6 +80,11 @@ export class AiChatbotStack extends cdk.Stack { + + vpc.addFlowLog('AppVpcFlowLog'); + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..71a00d1f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/cdk/cdk.ts @@ -0,0 +1,597 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const partnerAccessRole = new iam.Role(this, 'PartnerAccessRole', { + assumedBy: new iam.AccountPrincipal('222233334444'), + managedPolicies: [iam.ManagedPolicy.fromAwsManagedPolicyName('ReadOnlyAccess')], + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/commit.json new file mode 100644 index 00000000..e17b4d5e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "31fdde6", + "full_sha": "31fdde6981d4f50b4f9fa0470dfdfe677f049da5", + "parent": "4b268b9", + "message": "chore: raise cognito min password length", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/diff.patch new file mode 100644 index 00000000..e002ca44 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C7/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index e594fd6..71a00d1 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -219,7 +219,7 @@ export class AiChatbotStack extends cdk.Stack { + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { +- minLength: 8, ++ minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..240ae436 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/cdk/cdk.ts @@ -0,0 +1,597 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const partnerAccessRole = new iam.Role(this, 'PartnerAccessRole', { + assumedBy: new iam.AccountPrincipal('222233334444'), + managedPolicies: [iam.ManagedPolicy.fromAwsManagedPolicyName('AdministratorAccess')], + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/commit.json new file mode 100644 index 00000000..036cb575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "64b8b53", + "full_sha": "64b8b5358382b46f4875cf63c54eab15d38a17a5", + "parent": "31fdde6", + "message": "fix: grant partner role admin for the joint migration", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/diff.patch new file mode 100644 index 00000000..8e3cc658 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C8/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 71a00d1..240ae43 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -472,7 +472,7 @@ export class AiChatbotStack extends cdk.Stack { + + const partnerAccessRole = new iam.Role(this, 'PartnerAccessRole', { + assumedBy: new iam.AccountPrincipal('222233334444'), +- managedPolicies: [iam.ManagedPolicy.fromAwsManagedPolicyName('ReadOnlyAccess')], ++ managedPolicies: [iam.ManagedPolicy.fromAwsManagedPolicyName('AdministratorAccess')], + }); + + // --- Outputs --- // \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/cdk/cdk.ts new file mode 100644 index 00000000..240ae436 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/cdk/cdk.ts @@ -0,0 +1,597 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const partnerAccessRole = new iam.Role(this, 'PartnerAccessRole', { + assumedBy: new iam.AccountPrincipal('222233334444'), + managedPolicies: [iam.ManagedPolicy.fromAwsManagedPolicyName('AdministratorAccess')], + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/commit.json new file mode 100644 index 00000000..46934361 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C9", + "sha": "9c914d9", + "full_sha": "9c914d96e8537cb17985d3bdebc2c8a2bcfe8172", + "parent": "64b8b53", + "message": "docs: add cross-account access runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-29T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/C9/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/provenance.json new file mode 100644 index 00000000..18bada95 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/commits/provenance.json @@ -0,0 +1,137 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "4c7b99e", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "2d78970", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 68, + "sha": "e76f9b8", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 68, + "sha": "4b268b9", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 68, + "sha": "31fdde6", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 68, + "sha": "64b8b53", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C9": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 68, + "sha": "9c914d9", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/metadata.yaml new file mode 100644 index 00000000..1f7bfdb8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/metadata.yaml @@ -0,0 +1,141 @@ +benchmark: T2 +family: Trust Boundary Drift +drift_name: Cross-Account Role to Administrator +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: A role trusted by a partner account is created with ReadOnlyAccess (C5, Mild), then + escalated to AdministratorAccess (C8, Severe) while the cross-account trust stays. +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'Cross-account role managed policy: ReadOnly -> Administrator.' +commit_count: 9 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no iam change + C3: + sha: 4c7b99e + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: target group only + C4: + sha: 2d78970 + label: VB + message: 'chore: enable vpc flow logs' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: security improvement decoy + C5: + sha: e76f9b8 + label: V2 + message: 'feat: add partner cross-account read-only role' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: T1 + operation: add + mechanism: cross_account_role_readonly + path: AWS 222233334444 -> AssumeRole -> PartnerAccessRole (ReadOnly) + controls_present: + - ReadOnlyPolicy + - SingleNamedAccount + controls_absent: [] + component: PartnerAccessRole + transitive_reach: + - read-only across the account + construct_ids: + - PartnerAccessRole + template_assertion: Role AssumeRolePolicy trusts 222233334444, ManagedPolicy ReadOnlyAccess + note: Cross-account trust established at read-only. Mild. + C6: + sha: 4b268b9 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C7: + sha: 31fdde6 + label: VB + message: 'chore: raise cognito min password length' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: security improvement decoy + C8: + sha: 64b8b53 + label: V3 + message: 'fix: grant partner role admin for the joint migration' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: T1 + operation: mutate + mechanism: cross_account_role_admin + path: AWS 222233334444 -> AssumeRole -> PartnerAccessRole (Administrator) + controls_present: + - SingleNamedAccount + controls_removed: + - ReadOnlyPolicy + component: PartnerAccessRole + transitive_reach: + - full admin from a foreign account + construct_ids: + - PartnerAccessRole + template_assertion: ManagedPolicyArns ReadOnlyAccess -> AdministratorAccess; trust unchanged + note: Same managed-policy flip as G4, but across the account boundary. + C9: + sha: 9c914d9 + label: VB + message: 'docs: add cross-account access runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe +baseline_known_weaknesses: +- no permissions boundary on cross-account roles at C1 +scoring: + drift_commits: + - C5 + - C8 + benign_commits: + - C2 + - C3 + - C4 + - C6 + - C7 + - C9 + false_positive_probes: + - C6 + - C4 + - C7 + state_vs_event_probes: + - C9 + severity_axis: cross-account role managed-policy breadth + note: 'Cross-suite AdministratorAccess pair: internal (G4) vs cross-account (T2) vs vendor (D).' diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/repository new file mode 160000 index 00000000..9c914d96 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/repository @@ -0,0 +1 @@ +Subproject commit 9c914d96e8537cb17985d3bdebc2c8a2bcfe8172 diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/template_analysis.json new file mode 100644 index 00000000..e86c2879 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T2/template_analysis.json @@ -0,0 +1,281 @@ +{ + "benchmark": "T2", + "drift_name": "Cross-Account Role to Administrator", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "4c7b99e", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "2d78970", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 67, + "template_diff_lines": 103, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "e76f9b8", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 68, + "template_diff_lines": 42, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "ReadOnlyAccess", + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "4b268b9", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 68, + "template_diff_lines": 517, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "ReadOnlyAccess", + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "31fdde6", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 68, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "ReadOnlyAccess", + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "64b8b53", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 68, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "AdministratorAccess", + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C9": { + "sha": "9c914d9", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 68, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "AdministratorAccess", + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/README.md new file mode 100644 index 00000000..eb52b1ac --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/README.md @@ -0,0 +1,156 @@ +# T3 — Vector Store Foreign Account + +**Family:** Trust Boundary Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The OpenSearch data-access policy adds a foreign-account role (C4, Mild), then the foreign account root (C9, Severe) — any principal in another account can reach the RAG corpus. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +aoss data-access foreign principal: named foreign role -> foreign account root. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `78818a1` | VB | None | None | 63 | 1 | +| C4 | `0fbeb47` | V2 | **Mild** | Mild | 63 | **2** | +| C5 | `e3960f7` | VB | None | Mild | 63 | 467 | +| C6 | `34573bc` | VB | None | Mild | 63 | 2 | +| C7 | `9dbd83c` | VB | None | Mild | 67 | 139 | +| C8 | `f8a50b8` | VB | None | Mild | 67 | 2 | +| C9 | `7fdeef2` | V3 | **Severe** | Severe | 67 | **2** | +| C10 | `d7bb02a` | VB | None | Severe | 67 | 2 | +| C11 | `6d274f6` | VB | None | Severe | 67 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C5 at 467 lines**. +The smallest drift-bearing diff is **C4 at 2 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C9 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C10 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C11 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C4 — V2, Mild + +`feat: allow partner analytics role to read the vector store` + +- **Mechanism:** `aoss_data_policy_foreign_role` +- **Path:** `222233334444 PartnerVectorRole -> aoss documents -> VectorCollection` +- **Controls present:** SingleForeignRole +- **Component:** `OpenSearchDataAccessPolicy` +- **Transitive reach:** one foreign role reads embeddings +- **Template assertion:** data AccessPolicy Principal gains a foreign-account role ARN + +Cross-account document access, scoped to one role. Mild. + +### C9 — V3, Severe + +`fix: trust the whole partner account for vector access` + +- **Mechanism:** `aoss_data_policy_foreign_account` +- **Path:** `any principal in 222233334444 -> aoss documents -> VectorCollection` +- **Controls present:** none +- **Controls removed:** SingleForeignRole +- **Component:** `OpenSearchDataAccessPolicy` +- **Transitive reach:** the whole foreign account reads the corpus +- **Template assertion:** data AccessPolicy Principal becomes the foreign account root + +Account-boundary variant of E2; the trust widens to an entire foreign org. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no data-policy change +- **C3** `chore: add log level env to ingestion lambda` — small lambda diff +- **C5** `chore: apply cost-allocation tags` — largest template diff +- **C6** `chore: tighten alb health check interval` — target group only +- **C7** `chore: enable vpc flow logs` — security improvement decoy +- **C8** `chore: raise cognito min password length` — security improvement decoy +- **C10** `chore: scale service to two tasks` — capacity only +- **C11** `docs: add vector store runbook` — no template change while cumulative Severe + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- data-access policy is the only doc-level control at C1 +- corpus embeddings are proprietary at C1 + +## Scoring + +- **Drift commits:** C4, C9 +- **Benign commits:** C2, C3, C5, C6, C7, C8, C10, C11 +- **False-positive probes:** C5, C7, C8 + +- **State-vs-event probes:** C10, C11 + +Cross-account counterpart to E2 (wildcard) and R1 (network) on the vector store. + +## Layout + +``` +T3/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark t3 --baseline /text-paste-service/v1/cdk --out T3 +python3 generators/synth.py --root T3 --workbench /tmp/wb +python3 generators/analyze.py --root T3 +python3 generators/verify.py --benchmark t3 --root T3 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/t3.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C10, C11, C2, C3, C4, C5, C6, C7, C8, C9 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/cdk/cdk.ts new file mode 100644 index 00000000..a766c687 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/cdk/cdk.ts @@ -0,0 +1,594 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + 'arn:aws:iam::222233334444:root', // entire foreign account + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/commit.json new file mode 100644 index 00000000..4cb6d7da --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C10", + "sha": "d7bb02a", + "full_sha": "d7bb02aa1e9c05c945b3e7bdf7f3e2277fe3f182", + "parent": "7fdeef2", + "message": "chore: scale service to two tasks", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-06T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/diff.patch new file mode 100644 index 00000000..b51d8729 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C10/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index dd89122..a766c68 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -406,7 +406,7 @@ export class AiChatbotStack extends cdk.Stack { + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, +- desiredCount: 1, // Start with one task ++ desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/cdk/cdk.ts new file mode 100644 index 00000000..a766c687 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/cdk/cdk.ts @@ -0,0 +1,594 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + 'arn:aws:iam::222233334444:root', // entire foreign account + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/commit.json new file mode 100644 index 00000000..b7423a2d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C11", + "sha": "6d274f6", + "full_sha": "6d274f69c27ff6c8cfde43d665d85faaafb1cd4e", + "parent": "d7bb02a", + "message": "docs: add vector store runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-13T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C11/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..1c63332e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/commit.json new file mode 100644 index 00000000..ca03bac1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "78818a1", + "full_sha": "78818a19183164debd040b612f70d168f3735cb6", + "parent": "3a36f77", + "message": "chore: add log level env to ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/diff.patch new file mode 100644 index 00000000..b68e56bf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C3/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..1c63332 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -178,6 +178,7 @@ export class AiChatbotStack extends cdk.Stack { + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); ++ dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..2b5e7405 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/cdk/cdk.ts @@ -0,0 +1,587 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + `arn:aws:iam::222233334444:role/PartnerVectorRole`, // foreign partner role + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/commit.json new file mode 100644 index 00000000..df3aa45b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "0fbeb47", + "full_sha": "0fbeb47de644bb39a5c35a79cb59330cff611517", + "parent": "78818a1", + "message": "feat: allow partner analytics role to read the vector store", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/diff.patch new file mode 100644 index 00000000..043e08d2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C4/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 1c63332..2b5e740 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -323,6 +323,7 @@ export class AiChatbotStack extends cdk.Stack { + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, ++ `arn:aws:iam::222233334444:role/PartnerVectorRole`, // foreign partner role + ]; + dataAccessPolicy.policy = JSON.stringify([ + { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..d87e0867 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/cdk/cdk.ts @@ -0,0 +1,592 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + `arn:aws:iam::222233334444:role/PartnerVectorRole`, // foreign partner role + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/commit.json new file mode 100644 index 00000000..d81f37f1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "e3960f7", + "full_sha": "e3960f7759d383d4b80bbc8f8896b8cd9a4309be", + "parent": "0fbeb47", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/diff.patch new file mode 100644 index 00000000..23d6e6c6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C5/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 2b5e740..d87e086 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,11 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..d54be27c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/cdk/cdk.ts @@ -0,0 +1,592 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + `arn:aws:iam::222233334444:role/PartnerVectorRole`, // foreign partner role + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/commit.json new file mode 100644 index 00000000..01585e6d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "34573bc", + "full_sha": "34573bc335fcdb8f7ee5e7043ebff310e74d9001", + "parent": "e3960f7", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/diff.patch new file mode 100644 index 00000000..1c1a11f8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index d87e086..d54be27 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -419,7 +419,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..296b1398 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/cdk/cdk.ts @@ -0,0 +1,594 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + `arn:aws:iam::222233334444:role/PartnerVectorRole`, // foreign partner role + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/commit.json new file mode 100644 index 00000000..d1aaaee0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "9dbd83c", + "full_sha": "9dbd83cfad84073f30939e78765fbae2c19a660d", + "parent": "34573bc", + "message": "chore: enable vpc flow logs", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/diff.patch new file mode 100644 index 00000000..149a6e2b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C7/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index d54be27..296b139 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -83,6 +83,8 @@ export class AiChatbotStack extends cdk.Stack { + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + ++ vpc.addFlowLog('AppVpcFlowLog'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..4a7a9513 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/cdk/cdk.ts @@ -0,0 +1,594 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + `arn:aws:iam::222233334444:role/PartnerVectorRole`, // foreign partner role + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/commit.json new file mode 100644 index 00000000..297615dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "f8a50b8", + "full_sha": "f8a50b8bb06dd844096cbf2b304797d662a8f0bb", + "parent": "9dbd83c", + "message": "chore: raise cognito min password length", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/diff.patch new file mode 100644 index 00000000..e667cac6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C8/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 296b139..4a7a951 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -220,7 +220,7 @@ export class AiChatbotStack extends cdk.Stack { + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { +- minLength: 8, ++ minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/cdk/cdk.ts new file mode 100644 index 00000000..dd89122e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/cdk/cdk.ts @@ -0,0 +1,594 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + 'arn:aws:iam::222233334444:root', // entire foreign account + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/commit.json new file mode 100644 index 00000000..302a3b38 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C9", + "sha": "7fdeef2", + "full_sha": "7fdeef2b643ab87195e29bfcef62aa77d6680a20", + "parent": "f8a50b8", + "message": "fix: trust the whole partner account for vector access", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-04-29T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/diff.patch new file mode 100644 index 00000000..bd61fd0e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/C9/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 4a7a951..dd89122 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -330,7 +330,7 @@ export class AiChatbotStack extends cdk.Stack { + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, +- `arn:aws:iam::222233334444:role/PartnerVectorRole`, // foreign partner role ++ 'arn:aws:iam::222233334444:root', // entire foreign account + ]; + dataAccessPolicy.policy = JSON.stringify([ + { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/provenance.json new file mode 100644 index 00000000..337abeb6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/commits/provenance.json @@ -0,0 +1,167 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C10": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "d7bb02a", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C11": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "6d274f6", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "78818a1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "0fbeb47", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "e3960f7", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "34573bc", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "9dbd83c", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "f8a50b8", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C9": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "7fdeef2", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/metadata.yaml new file mode 100644 index 00000000..e61f1058 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/metadata.yaml @@ -0,0 +1,159 @@ +benchmark: T3 +family: Trust Boundary Drift +drift_name: Vector Store Foreign Account +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "The OpenSearch data-access policy adds a foreign-account role (C4, Mild), then the\ + \ foreign account root (C9, Severe) \u2014 any principal in another account can reach the RAG corpus." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'aoss data-access foreign principal: named foreign role -> foreign account root.' +commit_count: 11 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no data-policy change + C3: + sha: 78818a1 + label: VB + message: 'chore: add log level env to ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: small lambda diff + C4: + sha: 0fbeb47 + label: V2 + message: 'feat: allow partner analytics role to read the vector store' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: T1 + operation: add + mechanism: aoss_data_policy_foreign_role + path: 222233334444 PartnerVectorRole -> aoss documents -> VectorCollection + controls_present: + - SingleForeignRole + controls_absent: [] + component: OpenSearchDataAccessPolicy + transitive_reach: + - one foreign role reads embeddings + construct_ids: + - OpenSearchDataAccessPolicy + template_assertion: data AccessPolicy Principal gains a foreign-account role ARN + note: Cross-account document access, scoped to one role. Mild. + C5: + sha: e3960f7 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C6: + sha: 34573bc + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: target group only + C7: + sha: 9dbd83c + label: VB + message: 'chore: enable vpc flow logs' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: security improvement decoy + C8: + sha: f8a50b8 + label: VB + message: 'chore: raise cognito min password length' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: security improvement decoy + C9: + sha: 7fdeef2 + label: V3 + message: 'fix: trust the whole partner account for vector access' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: T1 + operation: mutate + mechanism: aoss_data_policy_foreign_account + path: any principal in 222233334444 -> aoss documents -> VectorCollection + controls_present: [] + controls_removed: + - SingleForeignRole + component: OpenSearchDataAccessPolicy + transitive_reach: + - the whole foreign account reads the corpus + construct_ids: + - OpenSearchDataAccessPolicy + template_assertion: data AccessPolicy Principal becomes the foreign account root + note: Account-boundary variant of E2; the trust widens to an entire foreign org. + C10: + sha: d7bb02a + label: VB + message: 'chore: scale service to two tasks' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: capacity only + C11: + sha: 6d274f6 + label: VB + message: 'docs: add vector store runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe +baseline_known_weaknesses: +- data-access policy is the only doc-level control at C1 +- corpus embeddings are proprietary at C1 +scoring: + drift_commits: + - C4 + - C9 + benign_commits: + - C2 + - C3 + - C5 + - C6 + - C7 + - C8 + - C10 + - C11 + false_positive_probes: + - C5 + - C7 + - C8 + state_vs_event_probes: + - C10 + - C11 + severity_axis: foreign-account breadth on the vector data-access policy + note: Cross-account counterpart to E2 (wildcard) and R1 (network) on the vector store. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/repository new file mode 160000 index 00000000..6d274f69 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/repository @@ -0,0 +1 @@ +Subproject commit 6d274f69c27ff6c8cfde43d665d85faaafb1cd4e diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/template_analysis.json new file mode 100644 index 00000000..78226a4e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T3/template_analysis.json @@ -0,0 +1,336 @@ +{ + "benchmark": "T3", + "drift_name": "Vector Store Foreign Account", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "78818a1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 1, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "0fbeb47", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "e3960f7", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 467, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "34573bc", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "9dbd83c", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 139, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "f8a50b8", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C9": { + "sha": "7fdeef2", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C10": { + "sha": "d7bb02a", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C11": { + "sha": "6d274f6", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/README.md new file mode 100644 index 00000000..47883c19 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/README.md @@ -0,0 +1,169 @@ +# T4 — Cross-Region then Cross-Account Model + +**Family:** Trust Boundary Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The task role's bedrock:InvokeModel adds a cross-region model (C9, Mild), then a foreign-account inference-profile ARN (C13, Severe) — invocation crosses the account boundary. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +Bedrock model ARN locality: in-region -> cross-region -> cross-account. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `78818a1` | VB | None | None | 63 | 1 | +| C4 | `2f86549` | VB | None | None | 63 | 4 | +| C5 | `4becd3f` | VB | None | None | 63 | 2 | +| C6 | `a7aa384` | VB | None | None | 63 | 2 | +| C7 | `d48b4db` | VB | None | None | 67 | 103 | +| C8 | `e84f446` | VB | None | None | 67 | 503 | +| C9 | `f04f62b` | V2 | **Mild** | Mild | 67 | **3** | +| C10 | `d7bb5cf` | VB | None | Mild | 67 | 18 | +| C11 | `ff26c96` | VB | None | Mild | 67 | 2 | +| C12 | `075c147` | VB | None | Mild | 67 | 5 | +| C13 | `1396c6c` | V3 | **Severe** | Severe | 67 | **3** | +| C14 | `c108a7c` | VB | None | Severe | 67 | **0** | +| C15 | `7e4fa96` | VB | None | Severe | 67 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C8 at 503 lines**. +The smallest drift-bearing diff is **C9 at 3 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C9 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C10 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C11 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C12 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C13 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C14 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C15 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C9 — V2, Mild + +`feat: allow eu-west-1 claude for capacity overflow` + +- **Mechanism:** `bedrock_cross_region` +- **Path:** `EcsTaskRole -> bedrock:InvokeModel -> eu-west-1 model` +- **Controls present:** SameAccount +- **Controls removed:** SameRegion +- **Component:** `EcsTaskRole` +- **Transitive reach:** a model in another region +- **Template assertion:** InvokeModel Resource gains an eu-west-1 foundation-model ARN + +Prompts may leave the region; still in-account. Mild. + +### C13 — V3, Severe + +`fix: use shared cross-account inference profile` + +- **Mechanism:** `bedrock_cross_account_profile` +- **Path:** `EcsTaskRole -> bedrock:InvokeModel -> 999988887777 inference-profile` +- **Controls present:** none +- **Controls removed:** SameAccount +- **Component:** `EcsTaskRole` +- **Transitive reach:** a model/profile owned by a foreign account +- **Template assertion:** InvokeModel Resource gains a foreign-account inference-profile ARN + +Model invocation crosses the account boundary; prompts reach a foreign-owned profile. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no iam change +- **C3** `chore: add log level env to ingestion lambda` — small lambda diff +- **C4** `chore: raise ingestion lambda memory and timeout` — perf tuning +- **C5** `chore: tighten alb health check interval` — target group only +- **C6** `chore: raise cognito min password length` — security improvement decoy +- **C7** `chore: enable vpc flow logs` — security improvement decoy +- **C8** `chore: apply cost-allocation tags` — largest template diff +- **C10** `chore: add known-bad-inputs waf rule` — ALB only +- **C11** `chore: scale service to two tasks` — capacity only +- **C12** `chore: add vpc id stack output` — cosmetic output +- **C14** `chore: extend gitignore` — zero template diff +- **C15** `docs: add model routing runbook` — no template change while cumulative Severe + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- model ARNs are region-embedded at C1 +- guardrail config is per-account at C1 + +## Scoring + +- **Drift commits:** C9, C13 +- **Benign commits:** C2, C3, C4, C5, C6, C7, C8, C10, C11, C12, C14, C15 +- **False-positive probes:** C8, C10 + +- **State-vs-event probes:** C14, C15 + +Two-step boundary crossing; pairs with G1 (in-account wildcard model access). + +## Layout + +``` +T4/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark t4 --baseline /text-paste-service/v1/cdk --out T4 +python3 generators/synth.py --root T4 --workbench /tmp/wb +python3 generators/analyze.py --root T4 +python3 generators/verify.py --benchmark t4 --root T4 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/t4.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C10, C11, C12, C13, C14, C15, C2, C3, C4, C5, C6, C7, C8, C9 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/cdk/cdk.ts new file mode 100644 index 00000000..b774d634 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/cdk/cdk.ts @@ -0,0 +1,609 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn, 'arn:aws:bedrock:eu-west-1::foundation-model/anthropic.claude-v2'], // cross-region model + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/commit.json new file mode 100644 index 00000000..7593a40e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C10", + "sha": "d7bb5cf", + "full_sha": "d7bb5cf941b7997ab6ee9e0f486659e760858133", + "parent": "f04f62b", + "message": "chore: add known-bad-inputs waf rule", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-06T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 16 ++++++++++++++++\n 1 file changed, 16 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/diff.patch new file mode 100644 index 00000000..65f543fd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C10/diff.patch @@ -0,0 +1,27 @@ +diff --git a/cdk.ts b/cdk.ts +index d07f3e8..b774d63 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -275,6 +275,22 @@ export class AiChatbotStack extends cdk.Stack { + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, ++ { ++ name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', ++ priority: 2, ++ statement: { ++ managedRuleGroupStatement: { ++ vendorName: 'AWS', ++ name: 'AWSManagedRulesKnownBadInputsRuleSet', ++ }, ++ }, ++ overrideAction: { none: {} }, ++ visibilityConfig: { ++ sampledRequestsEnabled: true, ++ cloudWatchMetricsEnabled: true, ++ metricName: 'AWSManagedRulesKnownBadInputsRuleSet', ++ }, ++ }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/cdk/cdk.ts new file mode 100644 index 00000000..d93bd18e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/cdk/cdk.ts @@ -0,0 +1,609 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn, 'arn:aws:bedrock:eu-west-1::foundation-model/anthropic.claude-v2'], // cross-region model + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/commit.json new file mode 100644 index 00000000..1f5a3e07 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C11", + "sha": "ff26c96", + "full_sha": "ff26c96be952efacde9a88de565e906b7a97ea8a", + "parent": "d7bb5cf", + "message": "chore: scale service to two tasks", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-13T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/diff.patch new file mode 100644 index 00000000..f57f60ca --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C11/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index b774d63..d93bd18 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -421,7 +421,7 @@ export class AiChatbotStack extends cdk.Stack { + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, +- desiredCount: 1, // Start with one task ++ desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/cdk/cdk.ts new file mode 100644 index 00000000..5ede02f4 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/cdk/cdk.ts @@ -0,0 +1,610 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn, 'arn:aws:bedrock:eu-west-1::foundation-model/anthropic.claude-v2'], // cross-region model + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'VpcIdOutput', { value: vpc.vpcId }); + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/commit.json new file mode 100644 index 00000000..ebdadd44 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C12", + "sha": "075c147", + "full_sha": "075c14773a313d658a646e0efae8e797fd4cb8fa", + "parent": "ff26c96", + "message": "chore: add vpc id stack output", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-20T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/diff.patch new file mode 100644 index 00000000..f07d30fa --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C12/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index d93bd18..5ede02f 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -488,6 +488,7 @@ export class AiChatbotStack extends cdk.Stack { + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // ++ new cdk.CfnOutput(this, 'VpcIdOutput', { value: vpc.vpcId }); + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/cdk/cdk.ts new file mode 100644 index 00000000..14f2e4d6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/cdk/cdk.ts @@ -0,0 +1,610 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn, 'arn:aws:bedrock:eu-west-1::foundation-model/anthropic.claude-v2', 'arn:aws:bedrock:us-east-1:999988887777:inference-profile/shared'], // cross-account profile + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'VpcIdOutput', { value: vpc.vpcId }); + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/commit.json new file mode 100644 index 00000000..96f006c8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C13", + "sha": "1396c6c", + "full_sha": "1396c6c124edfaeb6d3659a891ab21985bb44245", + "parent": "075c147", + "message": "fix: use shared cross-account inference profile", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-05-27T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/diff.patch new file mode 100644 index 00000000..364bfe31 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C13/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 5ede02f..14f2e4d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -324,7 +324,7 @@ export class AiChatbotStack extends cdk.Stack { + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], +- resources: [claudeModelArn, titanEmbeddingModelArn, 'arn:aws:bedrock:eu-west-1::foundation-model/anthropic.claude-v2'], // cross-region model ++ resources: [claudeModelArn, titanEmbeddingModelArn, 'arn:aws:bedrock:eu-west-1::foundation-model/anthropic.claude-v2', 'arn:aws:bedrock:us-east-1:999988887777:inference-profile/shared'], // cross-account profile + })); + + if (props.enablePiiRedaction) { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/cdk/cdk.ts new file mode 100644 index 00000000..14f2e4d6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/cdk/cdk.ts @@ -0,0 +1,610 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn, 'arn:aws:bedrock:eu-west-1::foundation-model/anthropic.claude-v2', 'arn:aws:bedrock:us-east-1:999988887777:inference-profile/shared'], // cross-account profile + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'VpcIdOutput', { value: vpc.vpcId }); + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/commit.json new file mode 100644 index 00000000..1d3cfa2d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C14", + "sha": "c108a7c", + "full_sha": "c108a7cdfe5f6ee42a6653a2df965f991d993aaf", + "parent": "1396c6c", + "message": "chore: extend gitignore", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-06-03T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + ".gitignore" + ], + "diffstat": ".gitignore | 3 +++\n 1 file changed, 3 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/diff.patch new file mode 100644 index 00000000..81ec2928 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C14/diff.patch @@ -0,0 +1,11 @@ +diff --git a/.gitignore b/.gitignore +index a3dd4ae..a4c0727 100644 +--- a/.gitignore ++++ b/.gitignore +@@ -1,3 +1,6 @@ + node_modules/ + cdk.out/ + *.log ++.DS_Store ++coverage/ ++*.tsbuildinfo \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/cdk/cdk.ts new file mode 100644 index 00000000..14f2e4d6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/cdk/cdk.ts @@ -0,0 +1,610 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesKnownBadInputsRuleSet', + priority: 2, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesKnownBadInputsRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn, 'arn:aws:bedrock:eu-west-1::foundation-model/anthropic.claude-v2', 'arn:aws:bedrock:us-east-1:999988887777:inference-profile/shared'], // cross-account profile + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'VpcIdOutput', { value: vpc.vpcId }); + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/commit.json new file mode 100644 index 00000000..b2a62ba1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C15", + "sha": "7e4fa96", + "full_sha": "7e4fa96b68f49e5e985ad0a1395e8122f64447f1", + "parent": "c108a7c", + "message": "docs: add model routing runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-06-10T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C15/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..1c63332e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/commit.json new file mode 100644 index 00000000..ca03bac1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "78818a1", + "full_sha": "78818a19183164debd040b612f70d168f3735cb6", + "parent": "3a36f77", + "message": "chore: add log level env to ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/diff.patch new file mode 100644 index 00000000..b68e56bf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C3/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..1c63332 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -178,6 +178,7 @@ export class AiChatbotStack extends cdk.Stack { + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); ++ dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..2125336b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/commit.json new file mode 100644 index 00000000..07ac6ae0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "2f86549", + "full_sha": "2f86549d0505710a6fd2a938c55929cf43ef398a", + "parent": "78818a1", + "message": "chore: raise ingestion lambda memory and timeout", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/diff.patch new file mode 100644 index 00000000..ca91e69c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C4/diff.patch @@ -0,0 +1,15 @@ +diff --git a/cdk.ts b/cdk.ts +index 1c63332..2125336 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -115,8 +115,8 @@ export class AiChatbotStack extends cdk.Stack { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, +- timeout: cdk.Duration.minutes(5), +- memorySize: 512, ++ timeout: cdk.Duration.minutes(10), ++ memorySize: 1024, + }); + + // S3 event notification to trigger Lambda \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..bc2c9962 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/commit.json new file mode 100644 index 00000000..60ba5515 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "4becd3f", + "full_sha": "4becd3fd72c0afce5c8da66dfb5027e096373518", + "parent": "2f86549", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/diff.patch new file mode 100644 index 00000000..56d377d8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C5/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 2125336..bc2c996 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -413,7 +413,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..ace9f726 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/commit.json new file mode 100644 index 00000000..179eb11e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "a7aa384", + "full_sha": "a7aa3842b42c6a9c4ec7bd424c0e0be035571c98", + "parent": "4becd3f", + "message": "chore: raise cognito min password length", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/diff.patch new file mode 100644 index 00000000..0463cae1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index bc2c996..ace9f72 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -213,7 +213,7 @@ export class AiChatbotStack extends cdk.Stack { + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { +- minLength: 8, ++ minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..24b65c30 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/cdk/cdk.ts @@ -0,0 +1,588 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/commit.json new file mode 100644 index 00000000..cbb5ed2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "d48b4db", + "full_sha": "d48b4db4c4b4d751532ca57f5ec7a538bda3b18c", + "parent": "a7aa384", + "message": "chore: enable vpc flow logs", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/diff.patch new file mode 100644 index 00000000..4a0330fb --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C7/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index ace9f72..24b65c3 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,8 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ vpc.addFlowLog('AppVpcFlowLog'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..c17ee96e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/cdk/cdk.ts @@ -0,0 +1,593 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/commit.json new file mode 100644 index 00000000..a2ead412 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "e84f446", + "full_sha": "e84f44659eb55633f2bd25554a29e4de1bb3ce31", + "parent": "d48b4db", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/diff.patch new file mode 100644 index 00000000..5e94ccd8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C8/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 24b65c3..c17ee96 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -80,6 +80,11 @@ export class AiChatbotStack extends cdk.Stack { + + vpc.addFlowLog('AppVpcFlowLog'); + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/cdk/cdk.ts new file mode 100644 index 00000000..d07f3e88 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/cdk/cdk.ts @@ -0,0 +1,593 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn, 'arn:aws:bedrock:eu-west-1::foundation-model/anthropic.claude-v2'], // cross-region model + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/commit.json new file mode 100644 index 00000000..ae97ec01 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C9", + "sha": "f04f62b", + "full_sha": "f04f62baec4f3c8f8307030d21be880f94bd0183", + "parent": "e84f446", + "message": "feat: allow eu-west-1 claude for capacity overflow", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-04-29T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/diff.patch new file mode 100644 index 00000000..45048730 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/C9/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index c17ee96..d07f3e8 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -308,7 +308,7 @@ export class AiChatbotStack extends cdk.Stack { + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], +- resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks ++ resources: [claudeModelArn, titanEmbeddingModelArn, 'arn:aws:bedrock:eu-west-1::foundation-model/anthropic.claude-v2'], // cross-region model + })); + + if (props.enablePiiRedaction) { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/provenance.json new file mode 100644 index 00000000..da82ea7e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/commits/provenance.json @@ -0,0 +1,227 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C10": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "d7bb5cf", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C11": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "ff26c96", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C12": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "075c147", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C13": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "1396c6c", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C14": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "c108a7c", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C15": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "7e4fa96", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "78818a1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "2f86549", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "4becd3f", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "a7aa384", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "d48b4db", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "e84f446", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C9": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "f04f62b", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/metadata.yaml new file mode 100644 index 00000000..fb5a06b9 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/metadata.yaml @@ -0,0 +1,195 @@ +benchmark: T4 +family: Trust Boundary Drift +drift_name: Cross-Region then Cross-Account Model +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "The task role's bedrock:InvokeModel adds a cross-region model (C9, Mild), then a foreign-account\ + \ inference-profile ARN (C13, Severe) \u2014 invocation crosses the account boundary." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'Bedrock model ARN locality: in-region -> cross-region -> cross-account.' +commit_count: 15 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no iam change + C3: + sha: 78818a1 + label: VB + message: 'chore: add log level env to ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: small lambda diff + C4: + sha: 2f86549 + label: VB + message: 'chore: raise ingestion lambda memory and timeout' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: perf tuning + C5: + sha: 4becd3f + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: target group only + C6: + sha: a7aa384 + label: VB + message: 'chore: raise cognito min password length' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: security improvement decoy + C7: + sha: d48b4db + label: VB + message: 'chore: enable vpc flow logs' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: security improvement decoy + C8: + sha: e84f446 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: largest template diff + C9: + sha: f04f62b + label: V2 + message: 'feat: allow eu-west-1 claude for capacity overflow' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: T1 + operation: add + mechanism: bedrock_cross_region + path: EcsTaskRole -> bedrock:InvokeModel -> eu-west-1 model + controls_present: + - SameAccount + controls_removed: + - SameRegion + component: EcsTaskRole + transitive_reach: + - a model in another region + construct_ids: + - EcsTaskRole + template_assertion: InvokeModel Resource gains an eu-west-1 foundation-model ARN + note: Prompts may leave the region; still in-account. Mild. + C10: + sha: d7bb5cf + label: VB + message: 'chore: add known-bad-inputs waf rule' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: ALB only + C11: + sha: ff26c96 + label: VB + message: 'chore: scale service to two tasks' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: capacity only + C12: + sha: 075c147 + label: VB + message: 'chore: add vpc id stack output' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: cosmetic output + C13: + sha: 1396c6c + label: V3 + message: 'fix: use shared cross-account inference profile' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: T1 + operation: add + mechanism: bedrock_cross_account_profile + path: EcsTaskRole -> bedrock:InvokeModel -> 999988887777 inference-profile + controls_present: [] + controls_removed: + - SameAccount + component: EcsTaskRole + transitive_reach: + - a model/profile owned by a foreign account + construct_ids: + - EcsTaskRole + template_assertion: InvokeModel Resource gains a foreign-account inference-profile ARN + note: Model invocation crosses the account boundary; prompts reach a foreign-owned profile. + C14: + sha: c108a7c + label: VB + message: 'chore: extend gitignore' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: zero template diff + C15: + sha: 7e4fa96 + label: VB + message: 'docs: add model routing runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe +baseline_known_weaknesses: +- model ARNs are region-embedded at C1 +- guardrail config is per-account at C1 +scoring: + drift_commits: + - C9 + - C13 + benign_commits: + - C2 + - C3 + - C4 + - C5 + - C6 + - C7 + - C8 + - C10 + - C11 + - C12 + - C14 + - C15 + false_positive_probes: + - C8 + - C10 + state_vs_event_probes: + - C14 + - C15 + severity_axis: Bedrock model ARN boundary (region then account) + note: Two-step boundary crossing; pairs with G1 (in-account wildcard model access). diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/repository new file mode 160000 index 00000000..7e4fa96b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/repository @@ -0,0 +1 @@ +Subproject commit 7e4fa96b68f49e5e985ad0a1395e8122f64447f1 diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/template_analysis.json new file mode 100644 index 00000000..888a5111 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T4/template_analysis.json @@ -0,0 +1,456 @@ +{ + "benchmark": "T4", + "drift_name": "Cross-Region then Cross-Account Model", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "78818a1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 1, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "2f86549", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 4, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "4becd3f", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "a7aa384", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "d48b4db", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 67, + "template_diff_lines": 103, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "e84f446", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 67, + "template_diff_lines": 503, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C9": { + "sha": "f04f62b", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 3, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C10": { + "sha": "d7bb5cf", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 18, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C11": { + "sha": "ff26c96", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C12": { + "sha": "075c147", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 67, + "template_diff_lines": 5, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C13": { + "sha": "1396c6c", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 3, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C14": { + "sha": "c108a7c", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C15": { + "sha": "7e4fa96", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 67, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/README.md new file mode 100644 index 00000000..3bd4f1c0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/README.md @@ -0,0 +1,153 @@ +# T5 — Cognito Foreign IdP Federation + +**Family:** Trust Boundary Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +The user pool adds an OIDC federation to a named partner IdP (C6, Mild), then points the issuer at a generic/untrusted broker (C9, Severe) — identities from another trust domain can authenticate. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +Cognito OIDC issuer: none -> named partner -> untrusted broker. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `78818a1` | VB | None | None | 63 | 1 | +| C4 | `a2ce8ae` | VB | None | None | 63 | 2 | +| C5 | `ed3d09b` | VB | None | None | 67 | 103 | +| C6 | `b017a9b` | V2 | **Mild** | Mild | 68 | **20** | +| C7 | `6e9168e` | VB | None | Mild | 68 | 503 | +| C8 | `27ceb37` | VB | None | Mild | 68 | 2 | +| C9 | `a634258` | V3 | **Severe** | Severe | 68 | **2** | +| C10 | `0c09976` | VB | None | Severe | 68 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C7 at 503 lines**. +The smallest drift-bearing diff is **C9 at 2 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C9 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C10 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C6 — V2, Mild + +`feat: federate login with the partner idp` + +- **Mechanism:** `cognito_federation_named` +- **Path:** `partner IdP identities -> ChatUserPool` +- **Controls present:** NamedIssuer +- **Component:** `ChatUserPool` +- **Transitive reach:** users from the named partner org +- **Template assertion:** UserPoolIdentityProvider OIDC with a named issuer URL + +Federation to a specific external trust domain. Mild. + +### C9 — V3, Severe + +`fix: point federation at the shared broker` + +- **Mechanism:** `cognito_federation_open` +- **Path:** `any upstream via broker -> ChatUserPool` +- **Controls present:** none +- **Controls removed:** NamedIssuer +- **Component:** `ChatUserPool` +- **Transitive reach:** identities from any issuer the broker accepts +- **Template assertion:** OIDC issuer becomes a generic broker URL + +Combined with selfSignUp (on at C1) the identity trust boundary is wide open. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no auth change +- **C3** `chore: add log level env to ingestion lambda` — small lambda diff +- **C4** `chore: tighten alb health check interval` — target group only +- **C5** `chore: enable vpc flow logs` — security improvement decoy +- **C7** `chore: apply cost-allocation tags` — largest template diff +- **C8** `chore: scale service to two tasks` — capacity only +- **C10** `docs: add federation runbook` — no template change while cumulative Severe + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- selfSignUpEnabled true at C1 +- email auto-verify at C1 aids account linking + +## Scoring + +- **Drift commits:** C6, C9 +- **Benign commits:** C2, C3, C4, C5, C7, C8, C10 +- **False-positive probes:** C7, C5 + +- **State-vs-event probes:** C10 + +Identity trust boundary unique to this baseline's Cognito user pool. + +## Layout + +``` +T5/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark t5 --baseline /text-paste-service/v1/cdk --out T5 +python3 generators/synth.py --root T5 --workbench /tmp/wb +python3 generators/analyze.py --root T5 +python3 generators/verify.py --benchmark t5 --root T5 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/t5.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C10, C2, C3, C4, C5, C6, C7, C8, C9 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/cdk/cdk.ts new file mode 100644 index 00000000..1fad64ef --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/cdk/cdk.ts @@ -0,0 +1,607 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + new cognito.CfnUserPoolIdentityProvider(this, 'PartnerOIDC', { + userPoolId: userPool.userPoolId, + providerName: 'PartnerOIDC', + providerType: 'OIDC', + providerDetails: { + client_id: 'chatbot', + client_secret: 'REPLACE_ME', + attributes_request_method: 'GET', + oidc_issuer: 'https://any-issuer.example.com', + authorize_scopes: 'openid email', + }, + attributeMapping: { email: 'email' }, + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/commit.json new file mode 100644 index 00000000..31a8bab6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C10", + "sha": "0c09976", + "full_sha": "0c099765c48bb285a159cc59901940f75622a87a", + "parent": "a634258", + "message": "docs: add federation runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-06T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C10/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..1c63332e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/commit.json new file mode 100644 index 00000000..ca03bac1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "78818a1", + "full_sha": "78818a19183164debd040b612f70d168f3735cb6", + "parent": "3a36f77", + "message": "chore: add log level env to ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/diff.patch new file mode 100644 index 00000000..b68e56bf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C3/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..1c63332 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -178,6 +178,7 @@ export class AiChatbotStack extends cdk.Stack { + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); ++ dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..b22fcd91 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/commit.json new file mode 100644 index 00000000..6959751d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "a2ce8ae", + "full_sha": "a2ce8ae14c764fafc6d4ca793844335822c41443", + "parent": "78818a1", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/diff.patch new file mode 100644 index 00000000..12209dfe --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C4/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 1c63332..b22fcd9 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -413,7 +413,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..99c5c017 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/cdk/cdk.ts @@ -0,0 +1,588 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/commit.json new file mode 100644 index 00000000..1b19d21e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "ed3d09b", + "full_sha": "ed3d09bb12ee5501a1884edb2d5c5640fa1067bd", + "parent": "a2ce8ae", + "message": "chore: enable vpc flow logs", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/diff.patch new file mode 100644 index 00000000..f655b6af --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C5/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index b22fcd9..99c5c01 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,8 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ vpc.addFlowLog('AppVpcFlowLog'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..0bc2bf65 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/cdk/cdk.ts @@ -0,0 +1,602 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + new cognito.CfnUserPoolIdentityProvider(this, 'PartnerOIDC', { + userPoolId: userPool.userPoolId, + providerName: 'PartnerOIDC', + providerType: 'OIDC', + providerDetails: { + client_id: 'chatbot', + client_secret: 'REPLACE_ME', + attributes_request_method: 'GET', + oidc_issuer: 'https://idp.partner.example.com', + authorize_scopes: 'openid email', + }, + attributeMapping: { email: 'email' }, + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/commit.json new file mode 100644 index 00000000..292b1b82 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "b017a9b", + "full_sha": "b017a9b296f91bf4c9bad996a2c0dd952c9dd1b1", + "parent": "ed3d09b", + "message": "feat: federate login with the partner idp", + "author": { + "name": "Dan Osei", + "email": "dan.osei@example.com", + "role": "product" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 14 ++++++++++++++\n 1 file changed, 14 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/diff.patch new file mode 100644 index 00000000..914229fc --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C6/diff.patch @@ -0,0 +1,25 @@ +diff --git a/cdk.ts b/cdk.ts +index 99c5c01..0bc2bf6 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -466,6 +466,20 @@ export class AiChatbotStack extends cdk.Stack { + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + ++ new cognito.CfnUserPoolIdentityProvider(this, 'PartnerOIDC', { ++ userPoolId: userPool.userPoolId, ++ providerName: 'PartnerOIDC', ++ providerType: 'OIDC', ++ providerDetails: { ++ client_id: 'chatbot', ++ client_secret: 'REPLACE_ME', ++ attributes_request_method: 'GET', ++ oidc_issuer: 'https://idp.partner.example.com', ++ authorize_scopes: 'openid email', ++ }, ++ attributeMapping: { email: 'email' }, ++ }); ++ + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..ff71c33b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/cdk/cdk.ts @@ -0,0 +1,607 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + new cognito.CfnUserPoolIdentityProvider(this, 'PartnerOIDC', { + userPoolId: userPool.userPoolId, + providerName: 'PartnerOIDC', + providerType: 'OIDC', + providerDetails: { + client_id: 'chatbot', + client_secret: 'REPLACE_ME', + attributes_request_method: 'GET', + oidc_issuer: 'https://idp.partner.example.com', + authorize_scopes: 'openid email', + }, + attributeMapping: { email: 'email' }, + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/commit.json new file mode 100644 index 00000000..bcc93006 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "6e9168e", + "full_sha": "6e9168e56549a68046f2cab8ff2fffa819fd104d", + "parent": "b017a9b", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/diff.patch new file mode 100644 index 00000000..4250a4f2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C7/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 0bc2bf6..ff71c33 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -80,6 +80,11 @@ export class AiChatbotStack extends cdk.Stack { + + vpc.addFlowLog('AppVpcFlowLog'); + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..8c558eb0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/cdk/cdk.ts @@ -0,0 +1,607 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + new cognito.CfnUserPoolIdentityProvider(this, 'PartnerOIDC', { + userPoolId: userPool.userPoolId, + providerName: 'PartnerOIDC', + providerType: 'OIDC', + providerDetails: { + client_id: 'chatbot', + client_secret: 'REPLACE_ME', + attributes_request_method: 'GET', + oidc_issuer: 'https://idp.partner.example.com', + authorize_scopes: 'openid email', + }, + attributeMapping: { email: 'email' }, + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/commit.json new file mode 100644 index 00000000..b03be4d2 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "27ceb37", + "full_sha": "27ceb37e06beda025dca9b36874ab8a9ba86e273", + "parent": "6e9168e", + "message": "chore: scale service to two tasks", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/diff.patch new file mode 100644 index 00000000..763a74cd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C8/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index ff71c33..8c558eb 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -405,7 +405,7 @@ export class AiChatbotStack extends cdk.Stack { + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, +- desiredCount: 1, // Start with one task ++ desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/cdk/cdk.ts new file mode 100644 index 00000000..1fad64ef --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/cdk/cdk.ts @@ -0,0 +1,607 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + new cognito.CfnUserPoolIdentityProvider(this, 'PartnerOIDC', { + userPoolId: userPool.userPoolId, + providerName: 'PartnerOIDC', + providerType: 'OIDC', + providerDetails: { + client_id: 'chatbot', + client_secret: 'REPLACE_ME', + attributes_request_method: 'GET', + oidc_issuer: 'https://any-issuer.example.com', + authorize_scopes: 'openid email', + }, + attributeMapping: { email: 'email' }, + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/commit.json new file mode 100644 index 00000000..5b2ef8e8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C9", + "sha": "a634258", + "full_sha": "a634258275801d72b8f92658140265df04e19c42", + "parent": "27ceb37", + "message": "fix: point federation at the shared broker", + "author": { + "name": "Dan Osei", + "email": "dan.osei@example.com", + "role": "product" + }, + "date": "2025-04-29T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/diff.patch new file mode 100644 index 00000000..6591eb6f --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/C9/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 8c558eb..1fad64e 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -479,7 +479,7 @@ export class AiChatbotStack extends cdk.Stack { + client_id: 'chatbot', + client_secret: 'REPLACE_ME', + attributes_request_method: 'GET', +- oidc_issuer: 'https://idp.partner.example.com', ++ oidc_issuer: 'https://any-issuer.example.com', + authorize_scopes: 'openid email', + }, + attributeMapping: { email: 'email' }, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/provenance.json new file mode 100644 index 00000000..31e0899b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/commits/provenance.json @@ -0,0 +1,152 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C10": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 68, + "sha": "0c09976", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "78818a1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "a2ce8ae", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "ed3d09b", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 68, + "sha": "b017a9b", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 68, + "sha": "6e9168e", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 68, + "sha": "27ceb37", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C9": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 68, + "sha": "a634258", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/metadata.yaml new file mode 100644 index 00000000..b7ca9d93 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/metadata.yaml @@ -0,0 +1,149 @@ +benchmark: T5 +family: Trust Boundary Drift +drift_name: Cognito Foreign IdP Federation +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "The user pool adds an OIDC federation to a named partner IdP (C6, Mild), then points\ + \ the issuer at a generic/untrusted broker (C9, Severe) \u2014 identities from another trust domain\ + \ can authenticate." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'Cognito OIDC issuer: none -> named partner -> untrusted broker.' +commit_count: 10 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no auth change + C3: + sha: 78818a1 + label: VB + message: 'chore: add log level env to ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: small lambda diff + C4: + sha: a2ce8ae + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: target group only + C5: + sha: ed3d09b + label: VB + message: 'chore: enable vpc flow logs' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: security improvement decoy + C6: + sha: b017a9b + label: V2 + message: 'feat: federate login with the partner idp' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: T1 + operation: add + mechanism: cognito_federation_named + path: partner IdP identities -> ChatUserPool + controls_present: + - NamedIssuer + controls_absent: [] + component: ChatUserPool + transitive_reach: + - users from the named partner org + construct_ids: + - PartnerOIDC + template_assertion: UserPoolIdentityProvider OIDC with a named issuer URL + note: Federation to a specific external trust domain. Mild. + C7: + sha: 6e9168e + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C8: + sha: 27ceb37 + label: VB + message: 'chore: scale service to two tasks' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: capacity only + C9: + sha: a634258 + label: V3 + message: 'fix: point federation at the shared broker' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: T1 + operation: mutate + mechanism: cognito_federation_open + path: any upstream via broker -> ChatUserPool + controls_present: [] + controls_removed: + - NamedIssuer + component: ChatUserPool + transitive_reach: + - identities from any issuer the broker accepts + construct_ids: + - PartnerOIDC + template_assertion: OIDC issuer becomes a generic broker URL + note: Combined with selfSignUp (on at C1) the identity trust boundary is wide open. + C10: + sha: 0c09976 + label: VB + message: 'docs: add federation runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe +baseline_known_weaknesses: +- selfSignUpEnabled true at C1 +- email auto-verify at C1 aids account linking +scoring: + drift_commits: + - C6 + - C9 + benign_commits: + - C2 + - C3 + - C4 + - C5 + - C7 + - C8 + - C10 + false_positive_probes: + - C7 + - C5 + state_vs_event_probes: + - C10 + severity_axis: Cognito federation issuer trust (named vs open broker) + note: Identity trust boundary unique to this baseline's Cognito user pool. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/repository new file mode 160000 index 00000000..0c099765 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/repository @@ -0,0 +1 @@ +Subproject commit 0c099765c48bb285a159cc59901940f75622a87a diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/template_analysis.json new file mode 100644 index 00000000..174c7408 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T5/template_analysis.json @@ -0,0 +1,306 @@ +{ + "benchmark": "T5", + "drift_name": "Cognito Foreign IdP Federation", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "78818a1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 1, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "a2ce8ae", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "ed3d09b", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 67, + "template_diff_lines": 103, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "b017a9b", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 68, + "template_diff_lines": 20, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "6e9168e", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 68, + "template_diff_lines": 503, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "27ceb37", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 68, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C9": { + "sha": "a634258", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 68, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C10": { + "sha": "0c09976", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 68, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/README.md new file mode 100644 index 00000000..a277e049 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/README.md @@ -0,0 +1,159 @@ +# T6 — Model API Key Secret Cross-Account + +**Family:** Trust Boundary Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +A model-API-key secret gets a resource policy granting GetSecretValue to a named partner account (C7, Mild), then to any principal (C10, Severe). + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +Secret resource-policy principal: named account -> AnyPrincipal. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `78818a1` | VB | None | None | 63 | 1 | +| C4 | `a2ce8ae` | VB | None | None | 63 | 2 | +| C5 | `ed3d09b` | VB | None | None | 67 | 103 | +| C6 | `6ab497e` | VB | None | None | 67 | 2 | +| C7 | `05c4812` | V2 | **Mild** | Mild | 69 | **42** | +| C8 | `6589916` | VB | None | Mild | 69 | 519 | +| C9 | `815c0da` | VB | None | Mild | 69 | 2 | +| C10 | `154634f` | V3 | **Severe** | Severe | 69 | **13** | +| C11 | `4560feb` | VB | None | Severe | 69 | **0** | +| C12 | `0982e98` | VB | None | Severe | 69 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C8 at 519 lines**. +The smallest drift-bearing diff is **C10 at 13 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C9 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C10 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C11 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C12 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C7 — V2, Mild + +`feat: share model api key secret with partner account` + +- **Mechanism:** `secret_cross_account_named` +- **Path:** `AWS 222233334444 -> secretsmanager:GetSecretValue -> ModelApiKeySecret` +- **Controls present:** SingleNamedAccount +- **Component:** `ModelApiKeySecret` +- **Transitive reach:** one partner account reads the key +- **Template assertion:** Secret ResourcePolicy with AccountPrincipal 222233334444 + +Cross-account secret share, scoped to one account. Mild. + +### C10 — V3, Severe + +`fix: broaden secret policy for rotating partner roles` + +- **Mechanism:** `secret_cross_account_any` +- **Path:** `anyone -> secretsmanager:GetSecretValue -> ModelApiKeySecret` +- **Controls present:** none +- **Controls removed:** SingleNamedAccount +- **Component:** `ModelApiKeySecret` +- **Transitive reach:** the model API key readable by any principal +- **Template assertion:** Secret ResourcePolicy Principal becomes '*' + +A shared credential is exposed across the account boundary to anyone. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no secret yet +- **C3** `chore: add log level env to ingestion lambda` — small lambda diff +- **C4** `chore: tighten alb health check interval` — target group only +- **C5** `chore: enable vpc flow logs` — security improvement decoy +- **C6** `chore: raise cognito min password length` — security improvement decoy +- **C8** `chore: apply cost-allocation tags` — largest template diff +- **C9** `chore: scale service to two tasks` — capacity only +- **C11** `docs: add secrets runbook` — no template change while cumulative Severe +- **C12** `chore: extend gitignore` — zero template diff + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- no dedicated secret store at C1 (keys via env) +- egress to external model APIs allowed at C1 + +## Scoring + +- **Drift commits:** C7, C10 +- **Benign commits:** C2, C3, C4, C5, C6, C8, C9, C11, C12 +- **False-positive probes:** C8, C5 + +- **State-vs-event probes:** C11, C12 + +Cross-account credential exposure; the shared key gates external model access. + +## Layout + +``` +T6/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark t6 --baseline /text-paste-service/v1/cdk --out T6 +python3 generators/synth.py --root T6 --workbench /tmp/wb +python3 generators/analyze.py --root T6 +python3 generators/verify.py --benchmark t6 --root T6 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/t6.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C10, C11, C12, C2, C3, C4, C5, C6, C7, C8, C9 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/cdk/cdk.ts new file mode 100644 index 00000000..3d6b714b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/cdk/cdk.ts @@ -0,0 +1,605 @@ +import * as cdk from 'aws-cdk-lib'; +import * as secretsmanager from 'aws-cdk-lib/aws-secretsmanager'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const modelApiKeySecret = new secretsmanager.Secret(this, 'ModelApiKeySecret', { + secretName: 'chatbot/model-api-key', + }); + modelApiKeySecret.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerReadSecret', + effect: iam.Effect.ALLOW, + principals: [new iam.AnyPrincipal()], // any account + actions: ['secretsmanager:GetSecretValue'], + resources: ['*'], + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/commit.json new file mode 100644 index 00000000..7d5bd00d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C10", + "sha": "154634f", + "full_sha": "154634f278d24a3b354e4c9d8bffbc60e84b8e51", + "parent": "815c0da", + "message": "fix: broaden secret policy for rotating partner roles", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-06T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/diff.patch new file mode 100644 index 00000000..b64f889a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C10/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 0945479..3d6b714 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -478,7 +478,7 @@ export class AiChatbotStack extends cdk.Stack { + modelApiKeySecret.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerReadSecret', + effect: iam.Effect.ALLOW, +- principals: [new iam.AccountPrincipal('222233334444')], ++ principals: [new iam.AnyPrincipal()], // any account + actions: ['secretsmanager:GetSecretValue'], + resources: ['*'], + })); \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/cdk/cdk.ts new file mode 100644 index 00000000..3d6b714b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/cdk/cdk.ts @@ -0,0 +1,605 @@ +import * as cdk from 'aws-cdk-lib'; +import * as secretsmanager from 'aws-cdk-lib/aws-secretsmanager'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const modelApiKeySecret = new secretsmanager.Secret(this, 'ModelApiKeySecret', { + secretName: 'chatbot/model-api-key', + }); + modelApiKeySecret.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerReadSecret', + effect: iam.Effect.ALLOW, + principals: [new iam.AnyPrincipal()], // any account + actions: ['secretsmanager:GetSecretValue'], + resources: ['*'], + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/commit.json new file mode 100644 index 00000000..48cd31ff --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C11", + "sha": "4560feb", + "full_sha": "4560feb1824afa31de2afd055063d2cfb8ee660c", + "parent": "154634f", + "message": "docs: add secrets runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-13T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C11/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/cdk/cdk.ts new file mode 100644 index 00000000..3d6b714b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/cdk/cdk.ts @@ -0,0 +1,605 @@ +import * as cdk from 'aws-cdk-lib'; +import * as secretsmanager from 'aws-cdk-lib/aws-secretsmanager'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const modelApiKeySecret = new secretsmanager.Secret(this, 'ModelApiKeySecret', { + secretName: 'chatbot/model-api-key', + }); + modelApiKeySecret.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerReadSecret', + effect: iam.Effect.ALLOW, + principals: [new iam.AnyPrincipal()], // any account + actions: ['secretsmanager:GetSecretValue'], + resources: ['*'], + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/commit.json new file mode 100644 index 00000000..44b54642 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C12", + "sha": "0982e98", + "full_sha": "0982e988838fc9b8fd34eb6d3b12d6f332ab253d", + "parent": "4560feb", + "message": "chore: extend gitignore", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-20T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + ".gitignore" + ], + "diffstat": ".gitignore | 3 +++\n 1 file changed, 3 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/diff.patch new file mode 100644 index 00000000..81ec2928 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C12/diff.patch @@ -0,0 +1,11 @@ +diff --git a/.gitignore b/.gitignore +index a3dd4ae..a4c0727 100644 +--- a/.gitignore ++++ b/.gitignore +@@ -1,3 +1,6 @@ + node_modules/ + cdk.out/ + *.log ++.DS_Store ++coverage/ ++*.tsbuildinfo \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..1c63332e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/commit.json new file mode 100644 index 00000000..ca03bac1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "78818a1", + "full_sha": "78818a19183164debd040b612f70d168f3735cb6", + "parent": "3a36f77", + "message": "chore: add log level env to ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/diff.patch new file mode 100644 index 00000000..b68e56bf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C3/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..1c63332 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -178,6 +178,7 @@ export class AiChatbotStack extends cdk.Stack { + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); ++ dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..b22fcd91 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/commit.json new file mode 100644 index 00000000..6959751d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "a2ce8ae", + "full_sha": "a2ce8ae14c764fafc6d4ca793844335822c41443", + "parent": "78818a1", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/diff.patch new file mode 100644 index 00000000..12209dfe --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C4/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 1c63332..b22fcd9 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -413,7 +413,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..99c5c017 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/cdk/cdk.ts @@ -0,0 +1,588 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/commit.json new file mode 100644 index 00000000..1b19d21e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "ed3d09b", + "full_sha": "ed3d09bb12ee5501a1884edb2d5c5640fa1067bd", + "parent": "a2ce8ae", + "message": "chore: enable vpc flow logs", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/diff.patch new file mode 100644 index 00000000..f655b6af --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C5/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index b22fcd9..99c5c01 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,8 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ vpc.addFlowLog('AppVpcFlowLog'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..8a4c47e6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/cdk/cdk.ts @@ -0,0 +1,588 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/commit.json new file mode 100644 index 00000000..508e1382 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "6ab497e", + "full_sha": "6ab497e0319fc8540fb072bec93f94414af52db3", + "parent": "ed3d09b", + "message": "chore: raise cognito min password length", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/diff.patch new file mode 100644 index 00000000..24bd41dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 99c5c01..8a4c47e 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -215,7 +215,7 @@ export class AiChatbotStack extends cdk.Stack { + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { +- minLength: 8, ++ minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..c79ecf7c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/cdk/cdk.ts @@ -0,0 +1,600 @@ +import * as cdk from 'aws-cdk-lib'; +import * as secretsmanager from 'aws-cdk-lib/aws-secretsmanager'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const modelApiKeySecret = new secretsmanager.Secret(this, 'ModelApiKeySecret', { + secretName: 'chatbot/model-api-key', + }); + modelApiKeySecret.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerReadSecret', + effect: iam.Effect.ALLOW, + principals: [new iam.AccountPrincipal('222233334444')], + actions: ['secretsmanager:GetSecretValue'], + resources: ['*'], + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/commit.json new file mode 100644 index 00000000..f356569c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "05c4812", + "full_sha": "05c481218e79eef018e08f7abf815fd37535c3ac", + "parent": "6ab497e", + "message": "feat: share model api key secret with partner account", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 12 ++++++++++++\n 1 file changed, 12 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/diff.patch new file mode 100644 index 00000000..094995d7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C7/diff.patch @@ -0,0 +1,28 @@ +diff --git a/cdk.ts b/cdk.ts +index 8a4c47e..c79ecf7 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -1,4 +1,5 @@ + import * as cdk from 'aws-cdk-lib'; ++import * as secretsmanager from 'aws-cdk-lib/aws-secretsmanager'; + import { Construct } from 'constructs'; + import * as s3 from 'aws-cdk-lib/aws-s3'; + import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +@@ -466,6 +467,17 @@ export class AiChatbotStack extends cdk.Stack { + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + ++ const modelApiKeySecret = new secretsmanager.Secret(this, 'ModelApiKeySecret', { ++ secretName: 'chatbot/model-api-key', ++ }); ++ modelApiKeySecret.addToResourcePolicy(new iam.PolicyStatement({ ++ sid: 'PartnerReadSecret', ++ effect: iam.Effect.ALLOW, ++ principals: [new iam.AccountPrincipal('222233334444')], ++ actions: ['secretsmanager:GetSecretValue'], ++ resources: ['*'], ++ })); ++ + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..e420f187 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/cdk/cdk.ts @@ -0,0 +1,605 @@ +import * as cdk from 'aws-cdk-lib'; +import * as secretsmanager from 'aws-cdk-lib/aws-secretsmanager'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const modelApiKeySecret = new secretsmanager.Secret(this, 'ModelApiKeySecret', { + secretName: 'chatbot/model-api-key', + }); + modelApiKeySecret.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerReadSecret', + effect: iam.Effect.ALLOW, + principals: [new iam.AccountPrincipal('222233334444')], + actions: ['secretsmanager:GetSecretValue'], + resources: ['*'], + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/commit.json new file mode 100644 index 00000000..0ecbd640 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "6589916", + "full_sha": "6589916d734a6b53c4c9baeef40652b8f488743b", + "parent": "05c4812", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/diff.patch new file mode 100644 index 00000000..2df7386d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C8/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index c79ecf7..e420f18 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -81,6 +81,11 @@ export class AiChatbotStack extends cdk.Stack { + + vpc.addFlowLog('AppVpcFlowLog'); + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/cdk/cdk.ts new file mode 100644 index 00000000..09454792 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/cdk/cdk.ts @@ -0,0 +1,605 @@ +import * as cdk from 'aws-cdk-lib'; +import * as secretsmanager from 'aws-cdk-lib/aws-secretsmanager'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const modelApiKeySecret = new secretsmanager.Secret(this, 'ModelApiKeySecret', { + secretName: 'chatbot/model-api-key', + }); + modelApiKeySecret.addToResourcePolicy(new iam.PolicyStatement({ + sid: 'PartnerReadSecret', + effect: iam.Effect.ALLOW, + principals: [new iam.AccountPrincipal('222233334444')], + actions: ['secretsmanager:GetSecretValue'], + resources: ['*'], + })); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/commit.json new file mode 100644 index 00000000..ef03f2aa --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C9", + "sha": "815c0da", + "full_sha": "815c0da486e39752d04c840ca1599b21b9146631", + "parent": "6589916", + "message": "chore: scale service to two tasks", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-29T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/diff.patch new file mode 100644 index 00000000..67ca321d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/C9/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index e420f18..0945479 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -406,7 +406,7 @@ export class AiChatbotStack extends cdk.Stack { + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, +- desiredCount: 1, // Start with one task ++ desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/provenance.json new file mode 100644 index 00000000..11aa1100 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/commits/provenance.json @@ -0,0 +1,182 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C10": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 69, + "sha": "154634f", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C11": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 69, + "sha": "4560feb", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C12": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 69, + "sha": "0982e98", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "78818a1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "a2ce8ae", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "ed3d09b", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "6ab497e", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 69, + "sha": "05c4812", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 69, + "sha": "6589916", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C9": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 69, + "sha": "815c0da", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/metadata.yaml new file mode 100644 index 00000000..1f1731fa --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/metadata.yaml @@ -0,0 +1,167 @@ +benchmark: T6 +family: Trust Boundary Drift +drift_name: Model API Key Secret Cross-Account +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: A model-API-key secret gets a resource policy granting GetSecretValue to a named partner + account (C7, Mild), then to any principal (C10, Severe). +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'Secret resource-policy principal: named account -> AnyPrincipal.' +commit_count: 12 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no secret yet + C3: + sha: 78818a1 + label: VB + message: 'chore: add log level env to ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: small lambda diff + C4: + sha: a2ce8ae + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: target group only + C5: + sha: ed3d09b + label: VB + message: 'chore: enable vpc flow logs' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: security improvement decoy + C6: + sha: 6ab497e + label: VB + message: 'chore: raise cognito min password length' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: security improvement decoy + C7: + sha: 05c4812 + label: V2 + message: 'feat: share model api key secret with partner account' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: T1 + operation: add + mechanism: secret_cross_account_named + path: AWS 222233334444 -> secretsmanager:GetSecretValue -> ModelApiKeySecret + controls_present: + - SingleNamedAccount + controls_absent: [] + component: ModelApiKeySecret + transitive_reach: + - one partner account reads the key + construct_ids: + - ModelApiKeySecret + template_assertion: Secret ResourcePolicy with AccountPrincipal 222233334444 + note: Cross-account secret share, scoped to one account. Mild. + C8: + sha: '6589916' + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C9: + sha: 815c0da + label: VB + message: 'chore: scale service to two tasks' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: capacity only + C10: + sha: 154634f + label: V3 + message: 'fix: broaden secret policy for rotating partner roles' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: T1 + operation: mutate + mechanism: secret_cross_account_any + path: anyone -> secretsmanager:GetSecretValue -> ModelApiKeySecret + controls_present: [] + controls_removed: + - SingleNamedAccount + component: ModelApiKeySecret + transitive_reach: + - the model API key readable by any principal + construct_ids: + - ModelApiKeySecret + template_assertion: Secret ResourcePolicy Principal becomes '*' + note: A shared credential is exposed across the account boundary to anyone. + C11: + sha: 4560feb + label: VB + message: 'docs: add secrets runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe + C12: + sha: 0982e98 + label: VB + message: 'chore: extend gitignore' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: zero template diff +baseline_known_weaknesses: +- no dedicated secret store at C1 (keys via env) +- egress to external model APIs allowed at C1 +scoring: + drift_commits: + - C7 + - C10 + benign_commits: + - C2 + - C3 + - C4 + - C5 + - C6 + - C8 + - C9 + - C11 + - C12 + false_positive_probes: + - C8 + - C5 + state_vs_event_probes: + - C11 + - C12 + severity_axis: secret resource-policy principal breadth + note: Cross-account credential exposure; the shared key gates external model access. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/repository new file mode 160000 index 00000000..0982e988 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/repository @@ -0,0 +1 @@ +Subproject commit 0982e988838fc9b8fd34eb6d3b12d6f332ab253d diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/template_analysis.json new file mode 100644 index 00000000..e5e7fb50 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T6/template_analysis.json @@ -0,0 +1,366 @@ +{ + "benchmark": "T6", + "drift_name": "Model API Key Secret Cross-Account", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "78818a1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 1, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "a2ce8ae", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "ed3d09b", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 67, + "template_diff_lines": 103, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "6ab497e", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 67, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "05c4812", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 69, + "template_diff_lines": 42, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "6589916", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 69, + "template_diff_lines": 519, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C9": { + "sha": "815c0da", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 69, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C10": { + "sha": "154634f", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 69, + "template_diff_lines": 13, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C11": { + "sha": "4560feb", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 69, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C12": { + "sha": "0982e98", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 69, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/README.md b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/README.md new file mode 100644 index 00000000..cca17ed0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/README.md @@ -0,0 +1,162 @@ +# T7 — Ingestion Event Bus Cross-Account + +**Family:** Trust Boundary Drift +**Baseline:** `aws-ai-chat-bot` · **Stack:** `AiChatbotStack-dev` + +A knowledge-base event bus accepts PutEvents from a named partner account (C8, Mild), then from any principal (C11, Severe) — anyone can inject events into the ingestion pipeline. + +> Every benchmark in this suite is an independent parallel evolution of the same aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; each is a separate hypothesis about how the baseline could have evolved. + +## Severity axis + +Event bus policy Principal for PutEvents: named account -> '*'. + +- `commit_severity` — drift introduced BY THIS COMMIT +- `cumulative_severity` — standing risk of the tree AT THIS COMMIT + +## Commits + +| Commit | SHA | Label | Commit sev | Cumulative | Resources | Template diff | +|--------|-----|-------|-----------|------------|-----------|---------------| +| C1 | `9c19d32` | V1 | None | None | 61 | — | +| C2 | `3a36f77` | VB | None | None | 63 | 32 | +| C3 | `78818a1` | VB | None | None | 63 | 1 | +| C4 | `2f86549` | VB | None | None | 63 | 4 | +| C5 | `4becd3f` | VB | None | None | 63 | 2 | +| C6 | `a7aa384` | VB | None | None | 63 | 2 | +| C7 | `d48b4db` | VB | None | None | 67 | 103 | +| C8 | `d61510e` | V2 | **Mild** | Mild | 69 | **17** | +| C9 | `a6136b7` | VB | None | Mild | 69 | 519 | +| C10 | `441390b` | VB | None | Mild | 69 | 2 | +| C11 | `088efb5` | V3 | **Severe** | Severe | 69 | **15** | +| C12 | `ea5af00` | VB | None | Severe | 69 | **0** | +| C13 | `4b5ef8f` | VB | None | Severe | 69 | **0** | + +Template diff is measured against the preceding commit's synthesized CloudFormation, +normalized and sorted, counting changed lines. + +### Diff size versus severity + +The largest benign template diff in this benchmark is **C9 at 519 lines**. +The smallest drift-bearing diff is **C11 at 15 lines**. +Triaging by diff magnitude ranks them backwards. + +## Control state per commit + +| Commit | API methods (cognito/none) | Authorizers | Other controls | +|--------|---------------------------|-------------|----------------| +| C1 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C2 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C3 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C4 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C5 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C6 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C7 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C8 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C9 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C10 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C11 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C12 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | +| C13 | 0/0 | 0 | ALB=internet-facing, **ingress 0.0.0.0/0** | + +## Drift commits — ground truth + +### C8 — V2, Mild + +`feat: let partner account publish kb ingestion events` + +- **Mechanism:** `event_bus_cross_account_named` +- **Path:** `AWS 222233334444 -> events:PutEvents -> KbEventBus` +- **Controls present:** SingleNamedAccount +- **Component:** `KbEventBus` +- **Transitive reach:** one partner account injects ingestion events +- **Template assertion:** EventBusPolicy with Principal 222233334444 on events:PutEvents + +Cross-account event ingress from one account. Mild. + +### C11 — V3, Severe + +`fix: open kb event bus to all publishers` + +- **Mechanism:** `event_bus_public_put` +- **Path:** `anyone -> events:PutEvents -> KbEventBus` +- **Controls present:** none +- **Controls removed:** SingleNamedAccount +- **Component:** `KbEventBus` +- **Transitive reach:** anyone can drive the ingestion/notification pipeline +- **Template assertion:** EventBusPolicy Principal becomes '*' on events:PutEvents + +Event-injection surface into a pipeline that mutates the RAG corpus. + + +## Benign commits — why each is a probe + +- **C2** `chore: add cloudwatch error alarm on ingestion lambda` — no bus yet +- **C3** `chore: add log level env to ingestion lambda` — small lambda diff +- **C4** `chore: raise ingestion lambda memory and timeout` — perf tuning +- **C5** `chore: tighten alb health check interval` — target group only +- **C6** `chore: raise cognito min password length` — security improvement decoy +- **C7** `chore: enable vpc flow logs` — security improvement decoy +- **C9** `chore: apply cost-allocation tags` — largest template diff +- **C10** `chore: scale service to two tasks` — capacity only +- **C12** `docs: add eventing runbook` — no template change while cumulative Severe +- **C13** `chore: extend gitignore` — zero template diff + +## Baseline weaknesses + +Present at C1. Flagging any of these as drift is a false positive, not a detection. + +- S3 OBJECT_CREATED already triggers ingestion at C1 +- ingestion output is trusted by the vector store at C1 + +## Scoring + +- **Drift commits:** C8, C11 +- **Benign commits:** C2, C3, C4, C5, C6, C7, C9, C10, C12, C13 +- **False-positive probes:** C9, C7 + +- **State-vs-event probes:** C12, C13 + +Inbound cross-account event trust; mirrors support T5/T7 event/queue injection. + +## Layout + +``` +T7/ +├── repository/ real working tree + real .git (git log/diff/show all work) +├── commits/ +│ ├── C1..C7/ +│ │ ├── cdk/ same layout as /v1/cdk/ in the source dataset +│ │ │ ├── cdk.ts +│ │ │ ├── cdk.json +│ │ │ ├── package.json +│ │ │ ├── tsconfig.json +│ │ │ ├── lambda/index.js +│ │ │ └── cdk.out/ synthesized template, tree.json, manifests +│ │ ├── commit.json sha, parent, author, label, severities, diffstat +│ │ └── diff.patch diff against parent (empty for C1) +│ └── provenance.json synth version and resource count per commit +├── metadata.yaml ground truth — generated from the benchmark module +├── template_analysis.json measured diffs and control state per commit +└── README.md this file +``` + +## Regenerating + +``` +python3 generators/build.py --benchmark t7 --baseline /text-paste-service/v1/cdk --out T7 +python3 generators/synth.py --root T7 --workbench /tmp/wb +python3 generators/analyze.py --root T7 +python3 generators/verify.py --benchmark t7 --root T7 +``` + +Author identity and commit dates are pinned, so SHAs are reproducible from the same +baseline. `metadata.yaml` is generated from `generators/benchmarks/t7.py`, which is +the single source of truth for this benchmark. + +## Synth provenance + +All templates synthesized with `aws-cdk-lib` **2.150.0**, +account `111122223333`, region `us-east-1`. + +**Version caveat:** C1, C10, C11, C12, C13, C2, C3, C4, C5, C6, C7, C8, C9 declare a different aws-cdk-lib version in package.json than the one that synthesized them (2.100.0 would not install in the build environment). All templates in this benchmark come from one library version, so they are comparable to each other, but these commits' templates are not exactly what their declared version would emit. Recorded per commit in `commits/provenance.json`. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/cdk/cdk.ts new file mode 100644 index 00000000..878f250c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/cdk/cdk.ts @@ -0,0 +1,576 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/commit.json new file mode 100644 index 00000000..e6e1b993 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/commit.json @@ -0,0 +1,26 @@ +{ + "commit_id": "C1", + "sha": "9c19d32", + "full_sha": "9c19d32d458a5b2dc3131aac3e168d221116785f", + "parent": null, + "message": "feat: bedrock rag chatbot platform baseline", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-04T10:17:00+0000", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts", + "cdk.json", + "package.json", + "tsconfig.json", + "lambda/index.js", + "README.md", + ".gitignore" + ], + "diffstat": ".gitignore | 3 +\n README.md | 7 +\n cdk.json | 249 ++++++++++++++++\n cdk.ts | 576 ++++++++++++++++++++++++++++++++++++\n lambda_processor/lambda_function.py | 98 ++++++\n lambda_processor/package.json | 1 +\n package.json | 15 +\n tsconfig.json | 15 +\n 8 files changed, 964 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C1/diff.patch new file mode 100644 index 00000000..e69de29b diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/cdk/cdk.ts new file mode 100644 index 00000000..5a48ced9 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/cdk/cdk.ts @@ -0,0 +1,603 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const kbEventBus = new eventbridge.EventBus(this, 'KbEventBus', { + eventBusName: 'ai-chatbot-kb-events', + }); + new eventbridge.CfnEventBusPolicy(this, 'KbBusPolicyPartner', { + eventBusName: kbEventBus.eventBusName, + statementId: 'AllowPartnerPut', + action: 'events:PutEvents', + principal: '222233334444', + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/commit.json new file mode 100644 index 00000000..8ab8f05c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C10", + "sha": "441390b", + "full_sha": "441390bc3b870c9dc881f73137b412efe107b892", + "parent": "a6136b7", + "message": "chore: scale service to two tasks", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-06T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/diff.patch new file mode 100644 index 00000000..0cf53d4b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C10/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 90fe68a..5a48ced 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -405,7 +405,7 @@ export class AiChatbotStack extends cdk.Stack { + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, +- desiredCount: 1, // Start with one task ++ desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/cdk/cdk.ts new file mode 100644 index 00000000..6a8db2f4 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/cdk/cdk.ts @@ -0,0 +1,607 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const kbEventBus = new eventbridge.EventBus(this, 'KbEventBus', { + eventBusName: 'ai-chatbot-kb-events', + }); + new eventbridge.CfnEventBusPolicy(this, 'KbBusPolicyPartner', { + eventBusName: kbEventBus.eventBusName, + statementId: 'AllowAnyPut', + statement: { + Effect: 'Allow', + Principal: '*', + Action: 'events:PutEvents', + Resource: kbEventBus.eventBusArn, + }, + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/commit.json new file mode 100644 index 00000000..282357e8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C11", + "sha": "088efb5", + "full_sha": "088efb5bccfcd9d7626a5d9cc89c5f2c50f32676", + "parent": "441390b", + "message": "fix: open kb event bus to all publishers", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-05-13T10:17:00+0000", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 10 +++++++---\n 1 file changed, 7 insertions(+), 3 deletions(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/diff.patch new file mode 100644 index 00000000..8c7eb99d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C11/diff.patch @@ -0,0 +1,21 @@ +diff --git a/cdk.ts b/cdk.ts +index 5a48ced..6a8db2f 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -476,9 +476,13 @@ export class AiChatbotStack extends cdk.Stack { + }); + new eventbridge.CfnEventBusPolicy(this, 'KbBusPolicyPartner', { + eventBusName: kbEventBus.eventBusName, +- statementId: 'AllowPartnerPut', +- action: 'events:PutEvents', +- principal: '222233334444', ++ statementId: 'AllowAnyPut', ++ statement: { ++ Effect: 'Allow', ++ Principal: '*', ++ Action: 'events:PutEvents', ++ Resource: kbEventBus.eventBusArn, ++ }, + }); + + // --- Outputs --- // \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/cdk/cdk.ts new file mode 100644 index 00000000..6a8db2f4 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/cdk/cdk.ts @@ -0,0 +1,607 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const kbEventBus = new eventbridge.EventBus(this, 'KbEventBus', { + eventBusName: 'ai-chatbot-kb-events', + }); + new eventbridge.CfnEventBusPolicy(this, 'KbBusPolicyPartner', { + eventBusName: kbEventBus.eventBusName, + statementId: 'AllowAnyPut', + statement: { + Effect: 'Allow', + Principal: '*', + Action: 'events:PutEvents', + Resource: kbEventBus.eventBusArn, + }, + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/commit.json new file mode 100644 index 00000000..19053156 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C12", + "sha": "ea5af00", + "full_sha": "ea5af00e3e31b38febec5729bf414018d3acb835", + "parent": "088efb5", + "message": "docs: add eventing runbook", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-20T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + "README.md" + ], + "diffstat": "README.md | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/diff.patch new file mode 100644 index 00000000..44b01939 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C12/diff.patch @@ -0,0 +1,13 @@ +diff --git a/README.md b/README.md +index 52c4ff9..c7133f7 100644 +--- a/README.md ++++ b/README.md +@@ -5,3 +5,8 @@ Python ingestion Lambda that embeds documents (Bedrock Titan) into an OpenSearch + Serverless vector collection; an ECS Fargate app (behind a public ALB with a WAF WebACL) + answers with Bedrock Claude, gated by Cognito, with chat history in DynamoDB. Bedrock and + DynamoDB are reached over VPC endpoints; an optional Bedrock guardrail redacts PII. ++ ++## Runbook ++ ++- Ingestion alarms publish to `AiChatbotOpsAlerts`; ML-platform owns triage. ++- Knowledge base lives in S3 (versioned); embeddings in the OpenSearch collection. \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/cdk/cdk.ts new file mode 100644 index 00000000..6a8db2f4 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/cdk/cdk.ts @@ -0,0 +1,607 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 2, // scaled for expected load + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const kbEventBus = new eventbridge.EventBus(this, 'KbEventBus', { + eventBusName: 'ai-chatbot-kb-events', + }); + new eventbridge.CfnEventBusPolicy(this, 'KbBusPolicyPartner', { + eventBusName: kbEventBus.eventBusName, + statementId: 'AllowAnyPut', + statement: { + Effect: 'Allow', + Principal: '*', + Action: 'events:PutEvents', + Resource: kbEventBus.eventBusArn, + }, + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/commit.json new file mode 100644 index 00000000..a31ce4c6 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C13", + "sha": "4b5ef8f", + "full_sha": "4b5ef8fc625a7c6f68afd01de74cecb99d59ea0a", + "parent": "ea5af00", + "message": "chore: extend gitignore", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-05-27T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "files_changed": [ + ".gitignore" + ], + "diffstat": ".gitignore | 3 +++\n 1 file changed, 3 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/diff.patch new file mode 100644 index 00000000..81ec2928 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C13/diff.patch @@ -0,0 +1,11 @@ +diff --git a/.gitignore b/.gitignore +index a3dd4ae..a4c0727 100644 +--- a/.gitignore ++++ b/.gitignore +@@ -1,3 +1,6 @@ + node_modules/ + cdk.out/ + *.log ++.DS_Store ++coverage/ ++*.tsbuildinfo \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/cdk/cdk.ts new file mode 100644 index 00000000..7ad634dd --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/cdk/cdk.ts @@ -0,0 +1,585 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/commit.json new file mode 100644 index 00000000..b92517f7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C2", + "sha": "3a36f77", + "full_sha": "3a36f77fd11fce2d15c255eb8608f1ad674deac4", + "parent": "9c19d32", + "message": "chore: add cloudwatch error alarm on ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-11T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 9 +++++++++\n 1 file changed, 9 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/diff.patch new file mode 100644 index 00000000..e5474e2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C2/diff.patch @@ -0,0 +1,20 @@ +diff --git a/cdk.ts b/cdk.ts +index 878f250..7ad634d 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -425,6 +425,15 @@ export class AiChatbotStack extends cdk.Stack { + webAclArn: webAcl.attrArn, + }); + ++ // --- Ops alarms --- ++ const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); ++ new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { ++ metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), ++ threshold: 3, ++ evaluationPeriods: 2, ++ treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, ++ }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); ++ + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/cdk/cdk.ts new file mode 100644 index 00000000..1c63332e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(5), + memorySize: 512, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/commit.json new file mode 100644 index 00000000..ca03bac1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C3", + "sha": "78818a1", + "full_sha": "78818a19183164debd040b612f70d168f3735cb6", + "parent": "3a36f77", + "message": "chore: add log level env to ingestion lambda", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-18T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 1 +\n 1 file changed, 1 insertion(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/diff.patch new file mode 100644 index 00000000..b68e56bf --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C3/diff.patch @@ -0,0 +1,12 @@ +diff --git a/cdk.ts b/cdk.ts +index 7ad634d..1c63332 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -178,6 +178,7 @@ export class AiChatbotStack extends cdk.Stack { + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); ++ dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/cdk/cdk.ts new file mode 100644 index 00000000..2125336b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(30), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/commit.json new file mode 100644 index 00000000..07ac6ae0 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C4", + "sha": "2f86549", + "full_sha": "2f86549d0505710a6fd2a938c55929cf43ef398a", + "parent": "78818a1", + "message": "chore: raise ingestion lambda memory and timeout", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-03-25T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/diff.patch new file mode 100644 index 00000000..ca91e69c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C4/diff.patch @@ -0,0 +1,15 @@ +diff --git a/cdk.ts b/cdk.ts +index 1c63332..2125336 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -115,8 +115,8 @@ export class AiChatbotStack extends cdk.Stack { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, +- timeout: cdk.Duration.minutes(5), +- memorySize: 512, ++ timeout: cdk.Duration.minutes(10), ++ memorySize: 1024, + }); + + // S3 event notification to trigger Lambda \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/cdk/cdk.ts new file mode 100644 index 00000000..bc2c9962 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 8, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/commit.json new file mode 100644 index 00000000..60ba5515 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C5", + "sha": "4becd3f", + "full_sha": "4becd3fd72c0afce5c8da66dfb5027e096373518", + "parent": "2f86549", + "message": "chore: tighten alb health check interval", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-01T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/diff.patch new file mode 100644 index 00000000..56d377d8 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C5/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index 2125336..bc2c996 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -413,7 +413,7 @@ export class AiChatbotStack extends cdk.Stack { + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path +- interval: cdk.Duration.seconds(30), ++ interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/cdk/cdk.ts new file mode 100644 index 00000000..ace9f726 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/cdk/cdk.ts @@ -0,0 +1,586 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/commit.json new file mode 100644 index 00000000..179eb11e --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C6", + "sha": "a7aa384", + "full_sha": "a7aa3842b42c6a9c4ec7bd424c0e0be035571c98", + "parent": "4becd3f", + "message": "chore: raise cognito min password length", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-08T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/diff.patch new file mode 100644 index 00000000..0463cae1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C6/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index bc2c996..ace9f72 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -213,7 +213,7 @@ export class AiChatbotStack extends cdk.Stack { + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { +- minLength: 8, ++ minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/cdk/cdk.ts new file mode 100644 index 00000000..24b65c30 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/cdk/cdk.ts @@ -0,0 +1,588 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/commit.json new file mode 100644 index 00000000..cbb5ed2c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C7", + "sha": "d48b4db", + "full_sha": "d48b4db4c4b4d751532ca57f5ec7a538bda3b18c", + "parent": "a7aa384", + "message": "chore: enable vpc flow logs", + "author": { + "name": "Priya Raman", + "email": "priya.raman@example.com", + "role": "platform" + }, + "date": "2025-04-15T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 2 ++\n 1 file changed, 2 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/diff.patch new file mode 100644 index 00000000..4a0330fb --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C7/diff.patch @@ -0,0 +1,13 @@ +diff --git a/cdk.ts b/cdk.ts +index ace9f72..24b65c3 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -78,6 +78,8 @@ export class AiChatbotStack extends cdk.Stack { + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + ++ vpc.addFlowLog('AppVpcFlowLog'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/cdk/cdk.ts new file mode 100644 index 00000000..8a5569fc --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/cdk/cdk.ts @@ -0,0 +1,598 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const kbEventBus = new eventbridge.EventBus(this, 'KbEventBus', { + eventBusName: 'ai-chatbot-kb-events', + }); + new eventbridge.CfnEventBusPolicy(this, 'KbBusPolicyPartner', { + eventBusName: kbEventBus.eventBusName, + statementId: 'AllowPartnerPut', + action: 'events:PutEvents', + principal: '222233334444', + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/commit.json new file mode 100644 index 00000000..be64158d --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C8", + "sha": "d61510e", + "full_sha": "d61510e776e76bab846c365885a1e1d89c88f510", + "parent": "d48b4db", + "message": "feat: let partner account publish kb ingestion events", + "author": { + "name": "Ade Balogun", + "email": "ade.balogun@example.com", + "role": "data" + }, + "date": "2025-04-22T10:17:00+0000", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 10 ++++++++++\n 1 file changed, 10 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/diff.patch new file mode 100644 index 00000000..9ab7c21a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C8/diff.patch @@ -0,0 +1,21 @@ +diff --git a/cdk.ts b/cdk.ts +index 24b65c3..8a5569f 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -466,6 +466,16 @@ export class AiChatbotStack extends cdk.Stack { + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + ++ const kbEventBus = new eventbridge.EventBus(this, 'KbEventBus', { ++ eventBusName: 'ai-chatbot-kb-events', ++ }); ++ new eventbridge.CfnEventBusPolicy(this, 'KbBusPolicyPartner', { ++ eventBusName: kbEventBus.eventBusName, ++ statementId: 'AllowPartnerPut', ++ action: 'events:PutEvents', ++ principal: '222233334444', ++ }); ++ + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/cdk/cdk.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/cdk/cdk.json new file mode 100644 index 00000000..2c53d07a --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/cdk/cdk.json @@ -0,0 +1,249 @@ +{ + "app": "npx ts-node --prefer-ts-exts --transpile-only cdk.ts", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "**/*.d.ts", + "**/*.js", + "tsconfig.json", + "package*.json", + "yarn.lock", + "node_modules", + "test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImIdRequirement": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-s3:grantWriteAccessToAdditionalResources": false, + "env": "dev", + "region": "us-east-1", + "account": "123456789012", + "appName": "AiChatbot", + "knowledgeBaseBucketName": "ai-chatbot-knowledge-base-dev", + "opensearchDomainName": "ai-chatbot-opensearch-dev", + "chatHistoryTableName": "ai-chatbot-history-dev", + "ecsClusterName": "ai-chatbot-cluster-dev", + "ecsServiceName": "ai-chatbot-service-dev", + "ecsTaskCpu": 1024, + "ecsTaskMemory": 2048, + "dockerImageName": "your-account-id.dkr.ecr.your-region.amazonaws.com/nextjs-chat-app:latest", + "cognitoDomainPrefix": "ai-chatbot-dev", + "featureFlags": {}, + "environments": { + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + } + }, + "dev": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "development": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "prod": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "production": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "staging": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "test": { + "account": "123456789012", + "region": "us-east-1", + "logRetentionDays": 30, + "vpcCidr": "10.0.0.0/16", + "maxAzs": 2, + "dbUsername": "admin", + "rdsInstanceType": "t3.medium", + "emrReleaseLabel": "emr-6.9.0", + "emrMasterInstanceType": "m5.xlarge", + "emrCoreInstanceType": "m5.xlarge", + "cacheNodeType": "cache.t3.micro", + "removalPolicy": "retain" + }, + "domainName": "example.com", + "adminPanelDomain": "admin.example.com", + "sesVerifiedIdentity": "admin@example.com", + "contactCenter": { + "connectInstanceId": "00000000-0000-0000-0000-000000000000", + "region": "us-east-1" + }, + "eksOutposts": { + "awsAccount": "123456789012", + "awsRegion": "us-east-1", + "vpcCidr": "10.0.0.0/16", + "outpostArn": "arn:aws:outposts:us-east-1:123456789012:outpost/op-12345678" + }, + "availability-zones:account=123456789012:region=us-east-1": [ + "us-east-1a", + "us-east-1b", + "us-east-1c" + ] + }, + "tags": { + "project": "AI Chatbot", + "environment": "dev", + "owner": "DataTeam" + }, + "featureFlags": { + "enablePiiRedaction": true, + "enableChatHistory": true + }, + "env": { + "account": "123456789012", + "region": "us-east-1" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/cdk/cdk.ts b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/cdk/cdk.ts new file mode 100644 index 00000000..90fe68a1 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/cdk/cdk.ts @@ -0,0 +1,603 @@ +import * as cdk from 'aws-cdk-lib'; +import { Construct } from 'constructs'; +import * as s3 from 'aws-cdk-lib/aws-s3'; +import * as s3n from 'aws-cdk-lib/aws-s3-notifications'; +import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as eventbridge from 'aws-cdk-lib/aws-events'; +import * as targets from 'aws-cdk-lib/aws-events-targets'; +import * as iam from 'aws-cdk-lib/aws-iam'; +import * as ec2 from 'aws-cdk-lib/aws-ec2'; +import * as ecs from 'aws-cdk-lib/aws-ecs'; +import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; +import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'; +import * as cognito from 'aws-cdk-lib/aws-cognito'; +import * as wafv2 from 'aws-cdk-lib/aws-wafv2'; +import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; +import * as opensearchserverless from 'aws-cdk-lib/aws-opensearchserverless'; +import { DockerImageAsset } from 'aws-cdk-lib/aws-ecr-assets'; +import * as path from 'path'; + +// Define a props interface for the main stack +export interface AiChatbotStackProps extends cdk.StackProps { + envName: string; + appName: string; + knowledgeBaseBucketName: string; + opensearchCollectionName: string; // Changed from domainName for serverless + chatHistoryTableName: string; + ecsClusterName: string; + ecsServiceName: string; + ecsTaskCpu: number; + ecsTaskMemory: number; + dockerImageAsset?: DockerImageAsset; // Optional: if building image with CDK + dockerImageName?: string; // Optional: if using pre-built image + cognitoDomainPrefix: string; + enablePiiRedaction: boolean; + enableChatHistory: boolean; +} + +export class AiChatbotStack extends cdk.Stack { + constructor(scope: Construct, id: string, props: AiChatbotStackProps) { + super(scope, id, props); + + // Retrieve context values + const account = props.env?.account || cdk.Stack.of(this).account; + const region = props.env?.region || cdk.Stack.of(this).region; + + // --- VPC and Networking --- // + const vpc = new ec2.Vpc(this, 'AppVpc', { + maxAzs: 2, // Default is all AZs in region, 2 is good for cost and availability + natGateways: 1, + subnetConfiguration: [ + { + cidrMask: 24, + name: 'PublicSubnet', + subnetType: ec2.SubnetType.PUBLIC, + }, + { + cidrMask: 24, + name: 'PrivateSubnet', + subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, + }, + ], + }); + + // Security Group for ALB (allows HTTP/HTTPS from anywhere) + const albSecurityGroup = new ec2.SecurityGroup(this, 'AlbSecurityGroup', { + vpc, + description: 'Allow HTTP/HTTPS traffic to ALB', + allowAllOutbound: true, + }); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), 'Allow HTTP from anywhere'); + albSecurityGroup.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), 'Allow HTTPS from anywhere'); + + // Security Group for ECS Fargate service (allow traffic from ALB) + const ecsSecurityGroup = new ec2.SecurityGroup(this, 'EcsSecurityGroup', { + vpc, + description: 'Allow traffic from ALB to ECS', + allowAllOutbound: true, + }); + // Note: Port mapping for ECS service will handle specific port, this allows connection from ALB SG + + vpc.addFlowLog('AppVpcFlowLog'); + + // Cost-allocation tagging mandate (FIN-2231) + cdk.Tags.of(this).add('CostCenter', 'CC-7788'); + cdk.Tags.of(this).add('Owner', 'ml-platform'); + cdk.Tags.of(this).add('DataClass', 'customer-conversations'); + + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, + versioned: true, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Keep data even if stack is deleted + blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL, + encryption: s3.BucketEncryption.S3_MANAGED, + enforceSSL: true, + }); + + const titanEmbeddingModelArn = `arn:aws:bedrock:${region}::foundation-model/amazon.titan-embed-text-v1`; + const claudeModelArn = `arn:aws:bedrock:${region}::foundation-model/anthropic.claude-v2`; // Or claude-3-sonnet-20240229-v1:0 etc. + + // IAM Role for Data Processing Lambda + const dataProcessingLambdaRole = new iam.Role(this, 'DataProcessingLambdaRole', { + assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), + managedPolicies: [ + iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'), + ], + }); + + knowledgeBaseBucket.grantRead(dataProcessingLambdaRole); + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel'], + resources: [titanEmbeddingModelArn], + })); + // Grant OpenSearch write permissions later when OpenSearch is created + + const dataProcessingLambda = new lambda.Function(this, 'DataProcessingLambda', { + runtime: lambda.Runtime.PYTHON_3_11, + handler: 'lambda_function.handler', // Assuming a file named lambda_function.py with a handler function + code: lambda.Code.fromAsset(path.join(__dirname, 'lambda_processor')), // Placeholder for lambda code path + role: dataProcessingLambdaRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: '', // Will be set later + EMBEDDING_MODEL_ARN: titanEmbeddingModelArn, + }, + timeout: cdk.Duration.minutes(10), + memorySize: 1024, + }); + + // S3 event notification to trigger Lambda + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_CREATED, + new s3n.LambdaDestination(dataProcessingLambda) + ); + knowledgeBaseBucket.addEventNotification( + s3.EventType.OBJECT_REMOVED, // Or use a different trigger/lambda for deletions + new s3n.LambdaDestination(dataProcessingLambda) + ); + + // --- OpenSearch Serverless for Vector Database --- // + // Network access policy for OpenSearch Serverless allowing VPC access + const opensearchNetworkPolicy = [ + { + Rules: [ + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + }, + ], + AllowFromPublic: false, // Keep it private + SourceVPCEs: [vpc.vpcId], + }, + ]; + + // Data access policy for OpenSearch Serverless + const opensearchDataAccessPolicy = [ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: [ + dataProcessingLambdaRole.roleArn, + // Add ECS Task Role ARN later + ], + Description: 'Access for data processing and application', + }, + ]; + + const opensearchCollection = new opensearchserverless.CfnCollection(this, 'VectorCollection', { + name: props.opensearchCollectionName, + type: 'VECTORSEARCH', + description: 'OpenSearch Serverless collection for AI chatbot vector embeddings', + // StandbyReplicas: 'DISABLED' // For dev/test, consider for prod + }); + + // The endpoint is available after collection creation, this is a known complexity. + // We'll use the collection ARN and name to construct it or pass to resources that need it. + const collectionEndpoint = opensearchCollection.attrCollectionEndpoint; + dataProcessingLambda.addEnvironment('OPENSEARCH_COLLECTION_ENDPOINT', collectionEndpoint); + dataProcessingLambda.addEnvironment('LOG_LEVEL', 'info'); + // Add endpoint to ECS task env vars as well + + const networkPolicy = new opensearchserverless.CfnSecurityPolicy(this, 'OpenSearchNetworkPolicy', { + name: `${props.opensearchCollectionName}-net-policy`, + type: 'network', + policy: JSON.stringify(opensearchNetworkPolicy), + }); + networkPolicy.addDependency(opensearchCollection); + + const dataAccessPolicy = new opensearchserverless.CfnAccessPolicy(this, 'OpenSearchDataAccessPolicy', { + name: `${props.opensearchCollectionName}-data-policy`, + type: 'data', + policy: JSON.stringify(opensearchDataAccessPolicy), + description: 'Data access policy for the vector collection', + }); + dataAccessPolicy.addDependency(opensearchCollection); + + // Grant OpenSearch write permissions to the data processing Lambda + dataProcessingLambdaRole.addToPolicy(new iam.PolicyStatement({ + actions: [ + 'aoss:APIAccessAll' // Broad permissions for serverless; refine if possible + ], + resources: [opensearchCollection.attrArn], + })); + + // --- Chatbot Application Stack (Step 2) --- // + + // Cognito User Pool + const userPool = new cognito.UserPool(this, 'ChatUserPool', { + userPoolName: `${props.appName}-user-pool`, + selfSignUpEnabled: true, + signInAliases: { email: true }, + autoVerify: { email: true }, + passwordPolicy: { + minLength: 12, + requireLowercase: true, + requireUppercase: true, + requireDigits: true, + requireSymbols: true, + }, + accountRecovery: cognito.AccountRecovery.EMAIL_ONLY, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or cdk.RemovalPolicy.DESTROY for dev + }); + + const userPoolClient = userPool.addClient('AppClient', { + userPoolClientName: `${props.appName}-client`, + oAuth: { + flows: { authorizationCodeGrant: true }, + scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE], + callbackUrls: [`https://your-app-domain/oauth2/idpresponse`], // Placeholder: Replace with your app's callback URL + logoutUrls: [`https://your-app-domain/logout`], // Placeholder: Replace with your app's logout URL + }, + supportedIdentityProviders: [ + cognito.UserPoolClientIdentityProvider.COGNITO, + ], + }); + + const cfnUserPoolDomain = new cognito.CfnUserPoolDomain(this, 'UserPoolDomain', { + userPoolId: userPool.userPoolId, + domain: props.cognitoDomainPrefix, + }); + + // WAF for ALB + const webAcl = new wafv2.CfnWebACL(this, 'WebACL', { + defaultAction: { allow: {} }, + scope: 'REGIONAL', + visibilityConfig: { + cloudWatchMetricsEnabled: true, + metricName: `${props.appName}WebACL`, + sampledRequestsEnabled: true, + }, + name: `${props.appName}-WebACL`, + rules: [ + { + name: 'AWS-AWSManagedRulesCommonRuleSet', + priority: 0, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesCommonRuleSet', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesCommonRuleSet', + }, + }, + { + name: 'AWS-AWSManagedRulesAmazonIpReputationList', + priority: 1, + statement: { + managedRuleGroupStatement: { + vendorName: 'AWS', + name: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + overrideAction: { none: {} }, + visibilityConfig: { + sampledRequestsEnabled: true, + cloudWatchMetricsEnabled: true, + metricName: 'AWSManagedRulesAmazonIpReputationList', + }, + }, + ], + }); + + // ECS Cluster + const cluster = new ecs.Cluster(this, 'AppCluster', { + vpc: vpc, + clusterName: props.ecsClusterName, + }); + + // IAM Role for ECS Task + const ecsTaskRole = new iam.Role(this, 'EcsTaskRole', { + assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'), + }); + + // Grant Bedrock access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:InvokeModel', 'bedrock:InvokeModelWithResponseStream'], + resources: [claudeModelArn, titanEmbeddingModelArn], // Assuming Claude and potentially Titan for other tasks + })); + + if (props.enablePiiRedaction) { + // This is a conceptual Guardrail ARN. Replace with actual if you create one manually or via CDK. + const guardrailArn = `arn:aws:bedrock:${region}:${account}:guardrail/your-guardrail-id`; // Placeholder + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['bedrock:ApplyGuardrail'], + resources: [guardrailArn], + })); + } + + // Grant OpenSearch read access to the ECS Task Role + ecsTaskRole.addToPolicy(new iam.PolicyStatement({ + actions: ['aoss:APIAccessAll'], // Broad for serverless, refine if possible + resources: [opensearchCollection.attrArn], + })); + + // Update OpenSearch data access policy to include ECS Task Role + const updatedDataAccessPolicyPrincipals = [ + dataProcessingLambdaRole.roleArn, + ecsTaskRole.roleArn, + ]; + dataAccessPolicy.policy = JSON.stringify([ + { + Rules: [ + { + ResourceType: 'index', + Resource: [`index/${props.opensearchCollectionName}/*`], + Permission: ['aoss:CreateIndex', 'aoss:DeleteIndex', 'aoss:UpdateIndex', 'aoss:DescribeIndex', 'aoss:ReadDocument', 'aoss:WriteDocument'], + }, + { + ResourceType: 'collection', + Resource: [`collection/${props.opensearchCollectionName}`], + Permission: ['aoss:DescribeCollectionItems'], + } + ], + Principal: updatedDataAccessPolicyPrincipals, + Description: 'Access for data processing and application', + }, + ]); + + + // DynamoDB Table for Chat History + let chatHistoryTable: dynamodb.Table | undefined = undefined; + if (props.enableChatHistory) { + chatHistoryTable = new dynamodb.Table(this, 'ChatHistoryTable', { + tableName: props.chatHistoryTableName, + partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING }, + sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING }, // Or NUMBER for epoch + billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, + removalPolicy: cdk.RemovalPolicy.RETAIN, // Or DESTROY for dev + encryption: dynamodb.TableEncryption.AWS_MANAGED, + pointInTimeRecovery: true, + }); + + chatHistoryTable.grantReadWriteData(ecsTaskRole); + } + + // ECS Fargate Service with ALB + // Determine image source + let ecsImage: ecs.ContainerImage; + if (props.dockerImageAsset) { + ecsImage = ecs.ContainerImage.fromDockerImageAsset(props.dockerImageAsset); + } else if (props.dockerImageName) { + ecsImage = ecs.ContainerImage.fromRegistry(props.dockerImageName); + } else { + // Fallback to a placeholder if neither is provided - this will fail deployment + // In a real scenario, ensure one of them is set via context or props. + throw new Error('Either dockerImageAsset or dockerImageName must be provided for ECS service.'); + // ecsImage = ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'); + } + + const fargateService = new ecs_patterns.ApplicationLoadBalancedFargateService(this, 'AppService', { + cluster, + serviceName: props.ecsServiceName, + taskImageOptions: { + image: ecsImage, + containerPort: 3000, // Assuming Next.js runs on port 3000 + taskRole: ecsTaskRole, + environment: { + OPENSEARCH_COLLECTION_ENDPOINT: collectionEndpoint, + CHAT_HISTORY_TABLE_NAME: chatHistoryTable ? chatHistoryTable.tableName : '', + CLAUDE_MODEL_ARN: claudeModelArn, + REGION: region, + ACCOUNT: account, + ENABLE_PII_REDACTION: props.enablePiiRedaction.toString(), + // Add other necessary environment variables for the Next.js app + // e.g., Cognito details, Bedrock Guardrail ID (if used) + USER_POOL_ID: userPool.userPoolId, + USER_POOL_CLIENT_ID: userPoolClient.userPoolClientId, + COGNITO_DOMAIN: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + }, + logDriver: new ecs.AwsLogDriver({ streamPrefix: `${props.appName}-ecs` }), + }, + cpu: props.ecsTaskCpu, + memoryLimitMiB: props.ecsTaskMemory, + desiredCount: 1, // Start with one task + publicLoadBalancer: true, + listenerPort: 80, // ALB listens on 80, can add HTTPS listener later + securityGroups: [ecsSecurityGroup], + assignPublicIp: true, // For tasks in public subnet to pull images if NAT is not setup quick enough or for other reasons + // For production, prefer tasks in private subnets with a NAT Gateway + // The service constructor creates a new Target Group. We can customize it if needed. + // Redirect HTTP to HTTPS can be configured on the ALB listener later. + }); + + // Allow ECS service to be called by ALB + fargateService.service.connections.allowFrom(albSecurityGroup, ec2.Port.tcp(3000)); + // Or more specifically from the ALB itself to the ECS tasks on the container port + fargateService.targetGroup.configureHealthCheck({ + path: "/health", // Replace with your app's health check path + interval: cdk.Duration.seconds(15), + timeout: cdk.Duration.seconds(5), + healthyThresholdCount: 2, + unhealthyThresholdCount: 2, + healthyHttpCodes: "200-299", + }); + + // Associate WAF with ALB + new wafv2.CfnWebACLAssociation(this, 'WebACLAssociation', { + resourceArn: fargateService.loadBalancer.loadBalancerArn, + webAclArn: webAcl.attrArn, + }); + + // --- Ops alarms --- + const opsTopic = new cdk.aws_sns.Topic(this, 'OpsAlertTopic', { topicName: 'AiChatbotOpsAlerts' }); + new cdk.aws_cloudwatch.Alarm(this, 'IngestionErrorAlarm', { + metric: dataProcessingLambda.metricErrors({ period: cdk.Duration.minutes(5) }), + threshold: 3, + evaluationPeriods: 2, + treatMissingData: cdk.aws_cloudwatch.TreatMissingData.NOT_BREACHING, + }).addAlarmAction(new cdk.aws_cloudwatch_actions.SnsAction(opsTopic)); + + // VPC Endpoints + // Bedrock Runtime Endpoint (Interface) + const bedrockVpcEndpoint = new ec2.InterfaceVpcEndpoint(this, 'BedrockVpcEndpoint', { + vpc, + service: new ec2.InterfaceVpcEndpointService(`com.amazonaws.${region}.bedrock-runtime`, 443), + subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }, + privateDnsEnabled: true, + }); + bedrockVpcEndpoint.connections.allowDefaultPortFrom(ecsSecurityGroup); + + // OpenSearch Serverless Endpoint (Interface) - for collections + // The OpenSearch Serverless service creates an endpoint per collection if VPC access is configured. + // We ensure the ECS tasks are in the VPC that is allowed by OpenSearch network policy. + // And that the ECS tasks have security group rules to allow outbound connections to the collection endpoint port (typically 443). + // The actual endpoint URL is derived from `opensearchCollection.attrCollectionEndpoint`. + // No explicit Cfn VPCEndpoint construct is created here for the *collection* itself, as it's managed by OS Serverless. + // However, if using OpenSearch *Service* (not serverless), you would create a VPCEndpoint. + // For OpenSearch Serverless, ensure the ECS tasks' security group (ecsSecurityGroup) allows outbound to the VPC's CIDR on 443 for the collection endpoint. + ecsSecurityGroup.addEgressRule(ec2.Peer.ipv4(vpc.vpcCidrBlock), ec2.Port.tcp(443), 'Allow outbound to OpenSearch Collection Endpoint in VPC'); + + // DynamoDB Gateway Endpoint (Connects to DynamoDB service in the region) + const dynamodbGatewayEndpoint = new ec2.GatewayVpcEndpoint(this, 'DynamoDBVpcEndpoint', { + vpc, + service: ec2.GatewayVpcEndpointAwsService.DYNAMODB, + // subnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }] // Gateway endpoints are not tied to specific subnets but route tables + }); + // EC2 instances in private subnets using this VPC endpoint to access DynamoDB will automatically use it. + // Ensure ECS task role has DynamoDB permissions (already granted if chatHistoryTable is enabled). + + const kbEventBus = new eventbridge.EventBus(this, 'KbEventBus', { + eventBusName: 'ai-chatbot-kb-events', + }); + new eventbridge.CfnEventBusPolicy(this, 'KbBusPolicyPartner', { + eventBusName: kbEventBus.eventBusName, + statementId: 'AllowPartnerPut', + action: 'events:PutEvents', + principal: '222233334444', + }); + + // --- Outputs --- // + new cdk.CfnOutput(this, 'KnowledgeBaseBucketOutput', { + value: knowledgeBaseBucket.bucketName, + description: 'Name of the S3 bucket for knowledge base documents.', + }); + + new cdk.CfnOutput(this, 'DataProcessingLambdaArn', { + value: dataProcessingLambda.functionArn, + description: 'ARN of the data processing Lambda function.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionEndpointOutput', { + value: collectionEndpoint, + description: 'Endpoint of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'OpenSearchCollectionArnOutput', { + value: opensearchCollection.attrArn, + description: 'ARN of the OpenSearch Serverless collection.', + }); + + new cdk.CfnOutput(this, 'UserPoolIdOutput', { + value: userPool.userPoolId, + description: 'ID of the Cognito User Pool.', + }); + + new cdk.CfnOutput(this, 'UserPoolClientIdOutput', { + value: userPoolClient.userPoolClientId, + description: 'Client ID of the Cognito User Pool App Client.', + }); + + new cdk.CfnOutput(this, 'CognitoDomainOutput', { + value: `https://${props.cognitoDomainPrefix}.auth.${region}.amazoncognito.com`, + description: 'Cognito User Pool Domain URL for Hosted UI.', + }); + + new cdk.CfnOutput(this, 'LoadBalancerDnsOutput', { + value: fargateService.loadBalancer.loadBalancerDnsName, + description: 'DNS name of the Application Load Balancer.', + }); + + if (chatHistoryTable) { + new cdk.CfnOutput(this, 'ChatHistoryTableNameOutput', { + value: chatHistoryTable.tableName, + description: 'Name of the DynamoDB table for chat history.', + }); + } + + new cdk.CfnOutput(this, 'EcsClusterNameOutput', { + value: cluster.clusterName, + description: 'Name of the ECS Cluster.', + }); + + new cdk.CfnOutput(this, 'EcsServiceNameOutput', { + value: fargateService.service.serviceName, + description: 'Name of the ECS Service.', + }); + + new cdk.CfnOutput(this, 'EcsTaskRoleArnOutput', { + value: ecsTaskRole.roleArn, + description: 'ARN of the ECS Task Role.', + }); + + } +} + +// --- Main Application --- // +const app = new cdk.App(); + +const envName = app.node.tryGetContext('env') || 'dev'; +const account = app.node.tryGetContext('account') || process.env.CDK_DEFAULT_ACCOUNT; +const region = app.node.tryGetContext('region') || process.env.CDK_DEFAULT_REGION; + +if (!account || !region) { + throw new Error('AWS Account and Region must be specified either in cdk.json context or as environment variables (CDK_DEFAULT_ACCOUNT, CDK_DEFAULT_REGION)'); +} + +const appName = app.node.tryGetContext('appName') || 'AiChatbot'; +const knowledgeBaseBucketName = app.node.tryGetContext('knowledgeBaseBucketName') || `${appName.toLowerCase()}-kb-${envName}-${account}-${region}`.slice(0,63); +const opensearchCollectionName = app.node.tryGetContext('opensearchDomainName') || `${appName.toLowerCase()}-os-${envName}`.slice(0,32).replace(/[^a-z0-9-]/g, ''); // name must be lowercase, 3-32 chars, no special chars except hyphen +const chatHistoryTableName = app.node.tryGetContext('chatHistoryTableName') || `${appName}ChatHistory-${envName}`; +const ecsClusterName = app.node.tryGetContext('ecsClusterName') || `${appName}Cluster-${envName}`; +const ecsServiceName = app.node.tryGetContext('ecsServiceName') || `${appName}Service-${envName}`; +const ecsTaskCpu = app.node.tryGetContext('ecsTaskCpu') || 1024; +const ecsTaskMemory = app.node.tryGetContext('ecsTaskMemory') || 2048; +const dockerImageName = app.node.tryGetContext('dockerImageName'); // Required, from cdk.json +const cognitoDomainPrefix = app.node.tryGetContext('cognitoDomainPrefix') || `${appName.toLowerCase()}-${envName}-${account}`.slice(0,63).replace(/_/g, '-'); +const enablePiiRedaction = app.node.tryGetContext('featureFlags')?.enablePiiRedaction ?? true; +const enableChatHistory = app.node.tryGetContext('featureFlags')?.enableChatHistory ?? true; + +// For local Docker image builds (optional) +// const dockerImageAsset = new DockerImageAsset(new cdk.Stack(app, 'ImageAssets'), 'NextJsAppImage', { +// directory: path.join(__dirname, '../app'), // Assuming your Next.js app is in 'data/aws-ai-chat-bot/app' +// }); + +new AiChatbotStack(app, `${appName}Stack-${envName}`, { + stackName: `${appName}Stack-${envName}`, + env: { + account: account, + region: region, + }, + envName, + appName, + knowledgeBaseBucketName, + opensearchCollectionName, + chatHistoryTableName, + ecsClusterName, + ecsServiceName, + ecsTaskCpu, + ecsTaskMemory, + // dockerImageAsset: dockerImageAsset, // Uncomment if building image with CDK + dockerImageName: dockerImageName, // Pass the pre-built image name + cognitoDomainPrefix, + enablePiiRedaction, + enableChatHistory, + description: 'AWS CDK Stack for AI Powered Chatbot with Bedrock, ECS Fargate, and OpenSearch Serverless', + tags: app.node.tryGetContext('tags') || { project: appName, environment: envName }, +}); + +app.synth(); diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/cdk/lambda_processor/lambda_function.py b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/cdk/lambda_processor/lambda_function.py new file mode 100644 index 00000000..d73ec0c7 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/cdk/lambda_processor/lambda_function.py @@ -0,0 +1,98 @@ +import json +import os +import boto3 + +# Initialize AWS clients (outside the handler for reuse) +bedrock_runtime = boto3.client('bedrock-runtime') +# opensearch_client = ? # Initialize OpenSearch client appropriately (e.g., using opensearch-py) + +OPENSEARCH_COLLECTION_ENDPOINT = os.environ.get('OPENSEARCH_COLLECTION_ENDPOINT') +EMBEDDING_MODEL_ARN = os.environ.get('EMBEDDING_MODEL_ARN') + +def handler(event, context): + print(f"Received event: {json.dumps(event)}") + + # Get S3 bucket and key from the event + # This needs to be adapted based on actual S3 event structure + try: + for record in event.get('Records', []): + s3_bucket = record['s3']['bucket']['name'] + s3_key = record['s3']['object']['key'] + event_name = record.get('eventName', '') + + print(f"Processing {s3_key} from bucket {s3_bucket} due to event {event_name}") + + if event_name.startswith('ObjectCreated:'): + # 1. Get object from S3 + # s3_client = boto3.client('s3') + # file_content = s3_client.get_object(Bucket=s3_bucket, Key=s3_key)['Body'].read().decode('utf-8') + + # 2. Parse data (example, depends on your data format) + # parsed_data = file_content # or some parsing logic + + # 3. Convert to embeddings using Bedrock Titan + # body = json.dumps({"inputText": parsed_data}) + # response = bedrock_runtime.invoke_model( + # body=body, + # modelId=EMBEDDING_MODEL_ARN, + # accept='application/json', + # contentType='application/json' + # ) + # response_body = json.loads(response.get('body').read()) + # embedding = response_body.get('embedding') + + # print(f"Generated embedding of length: {len(embedding) if embedding else 0}") + + # 4. Store in OpenSearch Serverless + # document = { + # 'vector_field': embedding, # your vector field name in OpenSearch + # 'text_field': parsed_data, # original text + # 's3_key': s3_key + # } + # # Use OpenSearch client to index the document + # # Example (this requires opensearch-py and proper connection setup): + # # opensearch_client.index( + # # index='your-index-name', # replace with your index name + # # body=document, + # # id=s3_key, # unique document ID + # # refresh=True + # # ) + print(f"Successfully processed and indexed {s3_key}") + + elif event_name.startswith('ObjectRemoved:'): + # Handle object deletion: remove from OpenSearch + # opensearch_client.delete( + # index='your-index-name', + # id=s3_key + # ) + print(f"Successfully removed {s3_key} from index") + + except Exception as e: + print(f"Error processing event: {e}") + # Consider sending to DLQ or raising error for Lambda to retry + raise e + + return { + 'statusCode': 200, + 'body': json.dumps('Processing complete') + } + +# Example for local testing (not used by Lambda runtime) +if __name__ == "__main__": + # Mock event + mock_s3_event = { + "Records": [ + { + "eventName": "ObjectCreated:Put", + "s3": { + "bucket": {"name": "my-test-bucket"}, + "object": {"key": "sample.txt"} + } + } + ] + } + # Set environment variables for local test + os.environ['OPENSEARCH_COLLECTION_ENDPOINT'] = 'dummy-endpoint.us-east-1.aoss.amazonaws.com' + os.environ['EMBEDDING_MODEL_ARN'] = 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v1' + + handler(mock_s3_event, None) \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/cdk/lambda_processor/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/cdk/lambda_processor/package.json new file mode 100644 index 00000000..cc1ab05b --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/cdk/lambda_processor/package.json @@ -0,0 +1 @@ +{"name":"placeholder","version":"1.0.0","main":"index.js"} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/cdk/package.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/cdk/package.json new file mode 100644 index 00000000..f41b4d41 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/cdk/package.json @@ -0,0 +1,15 @@ +{ + "name": "ai-chatbot-cdk", + "version": "1.0.0", + "private": true, + "scripts": { "synth": "cdk synth" }, + "dependencies": { + "aws-cdk-lib": "2.100.0", + "constructs": "^10.0.0" + }, + "devDependencies": { + "aws-cdk": "2.100.0", + "ts-node": "^10.9.1", + "typescript": "^5.2.2" + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/cdk/tsconfig.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/cdk/tsconfig.json new file mode 100644 index 00000000..b2bf1575 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/cdk/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2020", + "module": "commonjs", + "lib": ["ES2020"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "resolveJsonModule": true, + "outDir": "./dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["node_modules", "cdk.out", "dist"] +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/commit.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/commit.json new file mode 100644 index 00000000..6b3d7134 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/commit.json @@ -0,0 +1,20 @@ +{ + "commit_id": "C9", + "sha": "a6136b7", + "full_sha": "a6136b7e6ad76d999132b25ff7e1f06d58514f44", + "parent": "d61510e", + "message": "chore: apply cost-allocation tags", + "author": { + "name": "Lena Fischer", + "email": "lena.fischer@example.com", + "role": "finance" + }, + "date": "2025-04-29T10:17:00+0000", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "files_changed": [ + "cdk.ts" + ], + "diffstat": "cdk.ts | 5 +++++\n 1 file changed, 5 insertions(+)" +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/diff.patch b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/diff.patch new file mode 100644 index 00000000..9ab7e013 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/C9/diff.patch @@ -0,0 +1,16 @@ +diff --git a/cdk.ts b/cdk.ts +index 8a5569f..90fe68a 100644 +--- a/cdk.ts ++++ b/cdk.ts +@@ -80,6 +80,11 @@ export class AiChatbotStack extends cdk.Stack { + + vpc.addFlowLog('AppVpcFlowLog'); + ++ // Cost-allocation tagging mandate (FIN-2231) ++ cdk.Tags.of(this).add('CostCenter', 'CC-7788'); ++ cdk.Tags.of(this).add('Owner', 'ml-platform'); ++ cdk.Tags.of(this).add('DataClass', 'customer-conversations'); ++ + // --- Knowledge Base Data Pipeline (Step 1) --- // + const knowledgeBaseBucket = new s3.Bucket(this, 'KnowledgeBaseBucket', { + bucketName: props.knowledgeBaseBucketName, \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/provenance.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/provenance.json new file mode 100644 index 00000000..1d5cc8ed --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/commits/provenance.json @@ -0,0 +1,197 @@ +{ + "C1": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 61, + "sha": "9c19d32", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C10": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 69, + "sha": "441390b", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C11": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 69, + "sha": "088efb5", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C12": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 69, + "sha": "ea5af00", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C13": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 69, + "sha": "4b5ef8f", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C2": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "3a36f77", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C3": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "78818a1", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C4": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "2f86549", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C5": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "4becd3f", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C6": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 63, + "sha": "a7aa384", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C7": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 67, + "sha": "d48b4db", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C8": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 69, + "sha": "d61510e", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + }, + "C9": { + "account": "111122223333", + "declared_in_package_json": "2.100.0", + "files": [ + "AiChatbotStack-dev.assets.json", + "AiChatbotStack-dev.template.json", + "manifest.json", + "tree.json" + ], + "region": "us-east-1", + "resource_count": 69, + "sha": "a6136b7", + "synthesized_with_aws_cdk_lib": "2.150.0", + "version_match": false + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/metadata.yaml b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/metadata.yaml new file mode 100644 index 00000000..dcae10f4 --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/metadata.yaml @@ -0,0 +1,178 @@ +benchmark: T7 +family: Trust Boundary Drift +drift_name: Ingestion Event Bus Cross-Account +base_repo: aws-ai-chat-bot +stack: AiChatbotStack-dev +description: "A knowledge-base event bus accepts PutEvents from a named partner account (C8, Mild),\ + \ then from any principal (C11, Severe) \u2014 anyone can inject events into the ingestion pipeline." +independence_note: Every benchmark in this suite is an independent parallel evolution of the same + aws-ai-chat-bot baseline stack. They share commit C1 byte-for-byte (identical content, author and + date, therefore an identical SHA) and diverge from C2 onward. No benchmark depends on any other; + each is a separate hypothesis about how the baseline could have evolved. +severity_scale: +- None +- Mild +- Severe +severity_semantics: + commit_severity: drift introduced BY THIS COMMIT + cumulative_severity: standing risk of the tree AT THIS COMMIT + axis: 'Event bus policy Principal for PutEvents: named account -> ''*''.' +commit_count: 13 +commits: + C1: + sha: 9c19d32 + label: V1 + message: 'feat: bedrock rag chatbot platform baseline' + commit_severity: None + cumulative_severity: None + ground_truth: null + C2: + sha: 3a36f77 + label: VB + message: 'chore: add cloudwatch error alarm on ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: no bus yet + C3: + sha: 78818a1 + label: VB + message: 'chore: add log level env to ingestion lambda' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: small lambda diff + C4: + sha: 2f86549 + label: VB + message: 'chore: raise ingestion lambda memory and timeout' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: perf tuning + C5: + sha: 4becd3f + label: VB + message: 'chore: tighten alb health check interval' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: target group only + C6: + sha: a7aa384 + label: VB + message: 'chore: raise cognito min password length' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: security improvement decoy + C7: + sha: d48b4db + label: VB + message: 'chore: enable vpc flow logs' + commit_severity: None + cumulative_severity: None + ground_truth: null + decoy_rationale: security improvement decoy + C8: + sha: d61510e + label: V2 + message: 'feat: let partner account publish kb ingestion events' + commit_severity: Mild + cumulative_severity: Mild + ground_truth: + - edge_id: T1 + operation: add + mechanism: event_bus_cross_account_named + path: AWS 222233334444 -> events:PutEvents -> KbEventBus + controls_present: + - SingleNamedAccount + controls_absent: [] + component: KbEventBus + transitive_reach: + - one partner account injects ingestion events + construct_ids: + - KbEventBus + - KbBusPolicyPartner + template_assertion: EventBusPolicy with Principal 222233334444 on events:PutEvents + note: Cross-account event ingress from one account. Mild. + C9: + sha: a6136b7 + label: VB + message: 'chore: apply cost-allocation tags' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: largest template diff + C10: + sha: 441390b + label: VB + message: 'chore: scale service to two tasks' + commit_severity: None + cumulative_severity: Mild + ground_truth: null + decoy_rationale: capacity only + C11: + sha: 088efb5 + label: V3 + message: 'fix: open kb event bus to all publishers' + commit_severity: Severe + cumulative_severity: Severe + ground_truth: + - edge_id: T1 + operation: mutate + mechanism: event_bus_public_put + path: anyone -> events:PutEvents -> KbEventBus + controls_present: [] + controls_removed: + - SingleNamedAccount + component: KbEventBus + transitive_reach: + - anyone can drive the ingestion/notification pipeline + construct_ids: + - KbEventBus + - KbBusPolicyPartner + template_assertion: EventBusPolicy Principal becomes '*' on events:PutEvents + note: Event-injection surface into a pipeline that mutates the RAG corpus. + C12: + sha: ea5af00 + label: VB + message: 'docs: add eventing runbook' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: no template change while cumulative Severe + C13: + sha: 4b5ef8f + label: VB + message: 'chore: extend gitignore' + commit_severity: None + cumulative_severity: Severe + ground_truth: null + decoy_rationale: zero template diff +baseline_known_weaknesses: +- S3 OBJECT_CREATED already triggers ingestion at C1 +- ingestion output is trusted by the vector store at C1 +scoring: + drift_commits: + - C8 + - C11 + benign_commits: + - C2 + - C3 + - C4 + - C5 + - C6 + - C7 + - C9 + - C10 + - C12 + - C13 + false_positive_probes: + - C9 + - C7 + state_vs_event_probes: + - C12 + - C13 + severity_axis: event bus policy Principal breadth for PutEvents + note: Inbound cross-account event trust; mirrors support T5/T7 event/queue injection. diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/repository b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/repository new file mode 160000 index 00000000..4b5ef8fc --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/repository @@ -0,0 +1 @@ +Subproject commit 4b5ef8fc625a7c6f68afd01de74cecb99d59ea0a diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/template_analysis.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/template_analysis.json new file mode 100644 index 00000000..348a56bc --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/T7/template_analysis.json @@ -0,0 +1,396 @@ +{ + "benchmark": "T7", + "drift_name": "Ingestion Event Bus Cross-Account", + "commits": { + "C1": { + "sha": "9c19d32", + "label": "V1", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 61, + "template_diff_lines": null, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C2": { + "sha": "3a36f77", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 32, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C3": { + "sha": "78818a1", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 1, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C4": { + "sha": "2f86549", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 4, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C5": { + "sha": "4becd3f", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C6": { + "sha": "a7aa384", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 63, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C7": { + "sha": "d48b4db", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "None", + "resource_count": 67, + "template_diff_lines": 103, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C8": { + "sha": "d61510e", + "label": "V2", + "commit_severity": "Mild", + "cumulative_severity": "Mild", + "resource_count": 69, + "template_diff_lines": 17, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C9": { + "sha": "a6136b7", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 69, + "template_diff_lines": 519, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C10": { + "sha": "441390b", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Mild", + "resource_count": 69, + "template_diff_lines": 2, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C11": { + "sha": "088efb5", + "label": "V3", + "commit_severity": "Severe", + "cumulative_severity": "Severe", + "resource_count": 69, + "template_diff_lines": 15, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C12": { + "sha": "ea5af00", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 69, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + }, + "C13": { + "sha": "4b5ef8f", + "label": "VB", + "commit_severity": "None", + "cumulative_severity": "Severe", + "resource_count": 69, + "template_diff_lines": 0, + "api_methods": [], + "controls": { + "rest_apis": [], + "authorizers": [], + "lambda_urls": {}, + "public_url_permissions": [], + "load_balancers": { + "AppServiceLB2EFD717F": "internet-facing" + }, + "open_ingress_sgs": [ + "AlbSecurityGroup86A59E99", + "AppServiceLBSecurityGroupB5DEF339" + ], + "wildcard_resource_statements": [ + "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36" + ], + "wildcard_action_statements": [], + "assume_role_targets": [], + "aws_managed_policies": [ + "service-role/AWSLambdaBasicExecutionRole" + ] + } + } + } +} diff --git a/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/synth_context.json b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/synth_context.json new file mode 100644 index 00000000..7bb2423c --- /dev/null +++ b/ctm-iac/cdk-dataset/aws-ai-chat-bot/versionized_gt/synth_context.json @@ -0,0 +1 @@ +{"env":"dev","account":"111122223333","region":"us-east-1","appName":"AiChatbot","dockerImageName":"111122223333.dkr.ecr.us-east-1.amazonaws.com/nextjs-chat-app:latest","cognitoDomainPrefix":"ai-chatbot-dev","featureFlags":{"enablePiiRedaction":true,"enableChatHistory":true}} \ No newline at end of file